You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@polaris.apache.org by Alexandre Dutra <ad...@apache.org> on 2025/10/22 10:03:09 UTC
[DISCUSS] Polaris correlation IDs and telemetry
Hi all,
Today, Polaris has the notion of "request ID", but its purpose is not
entirely clear. It seems to serve as an observability feature to
facilitate correlation. A pending PR aims to rename it to "correlation
ID" for better alignment with industry standards [1].
However, this PR has brought to light overlaps with core telemetry
features: when OpenTelemetry (OTel) is enabled in Polaris, each
request already has a trace ID and span ID, making a separate
correlation ID redundant.
Moreover, using the OTel trace ID and span ID in Polaris events,
rather than the generated correlation ID, would significantly simplify
correlation of events with other traces.
Therefore, I propose the following changes:
1. If OTel is enabled, use the trace ID and span ID as the correlation
ID for the request, instead of generating a random correlation ID.
2. Otherwise, if a (Polaris-specific) correlation ID header is present
in the request, use it.
3. If neither of the above conditions is met, generate a random correlation ID.
I am somewhat undecided on the best approach when a correlation ID
header is present in the request. However, I believe it would be more
sensible to disregard it if OTel is enabled, as OTel offers a more
robust solution for client-to-server trace propagation, e.g. W3C Trace
Context propagation [2].
Please share your thoughts!
Thanks,
Alex
[1]: https://github.com/apache/polaris/pull/2757
[2]: https://www.w3.org/TR/trace-context
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Robert Stupp <sn...@snazy.de>.
Hi all,
it's preferable to have one correlation-ID, not two as today. So +1 on
using the OTel one.
Robert
On Wed, Oct 22, 2025 at 12:03 PM Alexandre Dutra <ad...@apache.org> wrote:
>
> Hi all,
>
> Today, Polaris has the notion of "request ID", but its purpose is not
> entirely clear. It seems to serve as an observability feature to
> facilitate correlation. A pending PR aims to rename it to "correlation
> ID" for better alignment with industry standards [1].
>
> However, this PR has brought to light overlaps with core telemetry
> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> request already has a trace ID and span ID, making a separate
> correlation ID redundant.
>
> Moreover, using the OTel trace ID and span ID in Polaris events,
> rather than the generated correlation ID, would significantly simplify
> correlation of events with other traces.
>
> Therefore, I propose the following changes:
>
> 1. If OTel is enabled, use the trace ID and span ID as the correlation
> ID for the request, instead of generating a random correlation ID.
> 2. Otherwise, if a (Polaris-specific) correlation ID header is present
> in the request, use it.
> 3. If neither of the above conditions is met, generate a random correlation ID.
>
> I am somewhat undecided on the best approach when a correlation ID
> header is present in the request. However, I believe it would be more
> sensible to disregard it if OTel is enabled, as OTel offers a more
> robust solution for client-to-server trace propagation, e.g. W3C Trace
> Context propagation [2].
>
> Please share your thoughts!
>
> Thanks,
> Alex
>
> [1]: https://github.com/apache/polaris/pull/2757
> [2]: https://www.w3.org/TR/trace-context
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi all,
I’m not sure if I misread some of the comments earlier - but I was under the impression that
> We could probably (additionally) store it [referring to: OTel Trace ID] in the request_id column if the Polaris-specific request ID header is not set.
meant that we would only persist one or the other, not both. Personally, the answer for the question I posed earlier still remains unclear to me: what is the use case for keeping both within events persistence? The reason for having them originally was to correlate events to requests that the user is sending. Is there any other use cases known (or on the horizon) at this time?
Based on what I’ve read, I believe we should only persist the single request ID that the server returned back to the client - regardless of whether it is the OTel Trace ID or the `x-request-id` header. As per Dmitri’s comment, I’m okay if we put in the additional_properties section whether the request ID was user-input or OTel, although I’m still not sure what that helps with?
Best,
Adnan Hemani
> On Oct 29, 2025, at 10:04 AM, Yufei Gu <fl...@gmail.com> wrote:
>
> I think the first question is whether any downstream use cases need to
> interpret the semantics of a request ID or OTel context. As I mentioned,
> the original purpose of a request ID was event correlation, e.g., tying a
> BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
> that’s the only requirement, an opaque correlation ID is good enough. We
> could change the field name(request-id) if needed.
>
> If we do have use cases that depend on the meaning of the request ID or
> OTel context, we can store them in additional_properties (
> https://www.google.com/url?q=https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%23L134-L134&source=gmail-imap&ust=1762362311000000&usg=AOvVaw04HH7BS9bFY4FOCh2ER0Pw).
> This is more flexible because it allows us to:
> 1. Record a key that indicates the format (e.g., x-request-id vs.
> otel-context).
> 2. Preserve both when Polaris receives both headers.
> 3. Add future tracing fields without a schema change.
>
> Yufei
>
>
> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
>> Hi All,
>>
>> From my POV the use cases for x-request-id and OTel are slightly different.
>> OTel is an observability use case where cross-linking of various pieces of
>> data is desired but not required in all cases (data loss is acceptable).
>> The x-request-id use case appears to have some very specific custom
>> applications without any obvious relationship to OTel.
>>
>> Given the current state of this discussion, I think it makes sense to
>> handle and persist them separately.
>>
>> However, if we intend x-request-id to act as an observability aid
>> alternative to OTel, then it might be preferable to have a config flag
>> indicating which one of them should be added to persistent event data. In
>> that case we should reuse the same column for them (and possibly add a
>> "type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...")
>>
>> WDYT?
>>
>> Thanks,
>> Dmitri.
>>
>>
>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org> wrote:
>>
>>> Hi Yufei,
>>>
>>>> I’d suggest persisting a single opaque correlation ID instead
>>>
>>> The problem I see with this is that readers cannot know the syntax of
>>> the opaque id, and will have to "probe" using some heuristics, e.g. if
>>> the id matches the OTel trace context syntax, asume it's OTel,
>>> otherwise... assume it's something else.
>>>
>>> That's not unfeasible, but I wonder if it isn't better to store the 2
>>> ids separately.
>>>
>>> Thanks,
>>> Alex
>>>
>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
>>>>
>>>> Thanks Alex for raising this discussion. Thanks everyone for chiming
>> in.
>>>> The plan looks good to me overall.
>>>>
>>>>> Update PolarisEvent, expose request ID if available, expose OTel
>>> context
>>>> if available.
>>>>
>>>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762362311000000&usg=AOvVaw11MsqGqjbsQVqTPLDZiOT2 persists both request id
>> and
>>>> otel context.
>>>>
>>>> Do we actually need to persist both?
>>>> From the original discussion, the intent of a request ID was to
>> correlate
>>>> multiple events generated by the same request, for example, linking a
>>>> BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we
>>>> really need is a single opaque identifier that ties related events
>>>> together, regardless of whether it comes from a request ID or an OTel
>>>> context.
>>>>
>>>> If we persist both fields (requestId and otelContext), it can lead to
>>>> consumer confusion, since downstream systems would need to decide which
>>> one
>>>> to rely on, potentially causing inconsistent event correlation. Unless
>>>> there are concrete use cases where event consumers need both, I’d
>> suggest
>>>> persisting a single opaque correlation ID instead. This keeps the
>> schema
>>>> simpler and ensures consistent event linking.
>>>>
>>>> Yufei
>>>>
>>>>
>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org>
>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Here is the PR: https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762362311000000&usg=AOvVaw11MsqGqjbsQVqTPLDZiOT2
>>>>>
>>>>> Thanks,
>>>>> Alex
>>>>>
>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
>> jb@nanthrax.net>
>>>>> wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Yeah it looks good to me and I agree we have a consensus.
>>>>>>
>>>>>> Thanks !
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org>
>> a
>>>>> écrit :
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Thanks for the good discussion so far. I think we have a broad
>>>>> consensus.
>>>>>>>
>>>>>>> Unless anybody disagrees I am going to move forward with the the
>>>>>>> following revised plan, incorporating the latest feedback:
>>>>>>>
>>>>>>> 1. Restore the original functionality for request IDs, change the
>>>>>>> default header name back to x-request-id
>>>>>>>
>>>>>>> 2. Remove RequestIdGenerator and related functionality.
>>>>>>>
>>>>>>> 3. Update PolarisEvent, expose request ID if available, expose
>> OTel
>>>>>>> context if available.
>>>>>>>
>>>>>>> 4. Update events table SQL schema: insert request ID if
>> available,
>>>>>>> insert OTel context if available.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Alex
>>>>>>>
>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
>>> jb@nanthrax.net
>>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I think it's better to use w3c_trace_context (better than two
>>>>> fields).
>>>>>>>>
>>>>>>>> Regarding x-request-id, I think it's for a different purpose,
>>>>>>>> specifically for persistence. Personally, I think we can
>> achieve
>>> the
>>>>>>>> same with span and events.
>>>>>>>> If we want to "distinguish" the two layers, it makes sense to
>>> have
>>>>>>>> x-request-id (not sure it's super helpful). Else, I think we
>> can
>>>>>>>> achieve the same with span/event.
>>>>>>>>
>>>>>>>> Just my $0.01
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested changes as
>>> well.
>>>>> Great
>>>>>>> find, Dmitri!
>>>>>>>>>
>>>>>>>>> I’m still debating with myself on whether we need to store
>> the
>>>>>>> `x-request-id` field as part of the Events persistence. Can we
>>> think
>>>>> of a
>>>>>>> good use case where this is more helpful to the user than the
>> OTel
>>>>>>> Trace/Span IDs? I am making the assumption here that those are
>>> still
>>>>> being
>>>>>>> returned back to the client.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Adnan Hemani
>>>>>>>>>
>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
>>> adutra@apache.org>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Dmitri,
>>>>>>>>>>
>>>>>>>>>> Yes, I think your suggestion to use just one field
>>>>> w3c_trace_context
>>>>>>>>>> makes more sense than two fields (span_id and trace_id).
>>>>>>>>>>
>>>>>>>>>> With that, I also think that we are slowly drifting into
>>>>>>>>>> implementation considerations; let's get consensus on the
>>> general
>>>>>>>>>> design first, and we can certainly fine-tune the actual
>> Java
>>>>> methods
>>>>>>>>>> and SQL table columns in the future PR. WDYT?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
>>>>>>> dimas@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> Many thanks for the background info, Adnan!
>>>>>>>>>>>
>>>>>>>>>>> +1 to action items proposed by Alex!
>>>>>>>>>>>
>>>>>>>>>>> Re: (3) Can we abstract request/trace info into a separate
>>>>> object
>>>>>>> without
>>>>>>>>>>> exposing those accessors on the Event class directly? OTel
>>>>> defines
>>>>>>>>>>> trace/span concepts, but in request ID is a bit foreign to
>>> OTel.
>>>>>>> Having
>>>>>>>>>>> tracing / request ID isolated in java could help with
>>>>> maintaining
>>>>>>> it and
>>>>>>>>>>> potentially supporting other (custom) tracing methods.
>>>>>>>>>>>
>>>>>>>>>>> Re: (4) I'd like to propose storing OTel correlation data
>>> in the
>>>>>>> form of a
>>>>>>>>>>> standard context propagation string, e.g. W3C
>> trace-context
>>> [1]
>>>>>>> (same value
>>>>>>>>>>> as its HTTP header), so the column could be called
>>>>>>> w3c_trace_context or
>>>>>>>>>>> simply trace_context.
>>>>>>>>>>>
>>>>>>>>>>> Open question: do we need to write a separate, individual
>>> trace
>>>>> ID
>>>>>>> field in
>>>>>>>>>>> SQL? I suppose it is not very useful since correlating it
>> to
>>>>> other
>>>>>>> trace
>>>>>>>>>>> data already requires understanding OTel context
>> propagation
>>>>> and a
>>>>>>> query
>>>>>>>>>>> against trace_context can still be made using
>>> string-matching
>>>>>>> clauses. We
>>>>>>>>>>> could probably (additionally) store it in the request_id
>>> column
>>>>> if
>>>>>>> the
>>>>>>>>>>> Polaris-specific request ID header is not set.
>>>>>>>>>>>
>>>>>>>>>>> As for span ID, I do not really see a use case for
>>> persisting it
>>>>>>>>>>> individually. It is very specific to OTel trace data
>>>>> construction.
>>>>>>>>>>>
>>>>>>>>>>> Actually, using the W3C trace context [1] encoding
>> probably
>>>>> makes
>>>>>>> sense in
>>>>>>>>>>> the java event representation too. Interested callers can
>>> easily
>>>>>>> decode
>>>>>>>>>>> this information since the format is well-defined. As a
>> side
>>>>>>> benefit, this
>>>>>>>>>>> opens opportunities for downstream event consumers to
>>> connect
>>>>>>> (propagate
>>>>>>>>>>> context) to traces that produced events based on the event
>>> data
>>>>>>> itself,
>>>>>>>>>>> without relying on the intermediate frameworks. This may
>> be
>>>>>>> desirable since
>>>>>>>>>>> the current Polaris event persistence impl. writes events
>> in
>>>>>>> batches, so
>>>>>>>>>>> the association to individual requests that produced those
>>>>> events
>>>>>>> is lost.
>>>>>>>>>>> Whether to perform this kind of context propagation will
>> be
>>> at
>>>>> the
>>>>>>>>>>> discretion of the event consumer, of course (outside of
>>> Polaris
>>>>>>> code).
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context/%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2fn0lRsTx-f9r4PCz8wmJK&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2dvj8jJ6YcldfV2g-Xc27v
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Dmitri.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
>>>>> adutra@apache.org>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for chiming in; the context around request IDs
>>> is now
>>>>>>> clear.
>>>>>>>>>>>>
>>>>>>>>>>>> Trying to summarize this thread into actionable items,
>>> here's
>>>>> what
>>>>>>> I
>>>>>>>>>>>> propose:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Restore the original functionality for request IDs.
>>>>>>>>>>>> - Change the default header name back to x-request-id
>>>>> (despite
>>>>>>> the
>>>>>>>>>>>> x- prefix being deprecated), but keep it configurable as
>>> today.
>>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
>>>>>>>>>>>> - Do not generate a request ID if the REST client
>>> doesn't
>>>>>>> provide one.
>>>>>>>>>>>> 3. Update PolarisEvent:
>>>>>>>>>>>> - Expose new requestId(), traceId(), spanId() methods,
>>> all
>>>>>>> nullable.
>>>>>>>>>>>> - This would align with the emerging consensus around
>>>>> including
>>>>>>>>>>>> contextual information in PolarisEvent [1].
>>>>>>>>>>>> 4. Update events table SQL schema:
>>>>>>>>>>>> - Insert the client-provided request ID into the
>>> request_id
>>>>>>>>>>>> column; otherwise, insert null.
>>>>>>>>>>>> - Add two new nullable columns, trace_id and span_id,
>>> and
>>>>>>> populate
>>>>>>>>>>>> them if OTel is enabled.
>>>>>>>>>>>>
>>>>>>>>>>>> From our discussions, I think it's important to not
>>> conflate
>>>>> OTel
>>>>>>>>>>>> tracing fields with Envoy tracing fields, which is why I
>>>>> suggest we
>>>>>>>>>>>> use separate fields / columns for them.
>>>>>>>>>>>>
>>>>>>>>>>>> Would the above plan work for everyone?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Alex
>>>>>>>>>>>>
>>>>>>>>>>>> [1]:
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw02wPvb0qxRzYAKEP0h8l9T&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2T5pohrqE8tGS2t3tyP2CP
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks to Alex for starting this thread - because of
>>> this, I’m
>>>>>>> just
>>>>>>>>>>>> coming to the realization that OTel Trace and Span IDs
>> are
>>>>> coming
>>>>>>> built-in
>>>>>>>>>>>> with Quarkus and my previous work to generate a
>>>>>>> Request/Correlation ID is
>>>>>>>>>>>> likely not needed as a result. My original motivation for
>>>>>>> generation of a
>>>>>>>>>>>> Request/Correlation ID was to ensure that any client can
>>>>> uniquely
>>>>>>> identify
>>>>>>>>>>>> a request made to Polaris, which would be especially
>>> useful for
>>>>>>> debugging
>>>>>>>>>>>> failing requests or identifying call patterns.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we should
>>> remove
>>>>> the
>>>>>>>>>>>> Request/Correlation ID generation and always use the OTel
>>>>>>> trace/span IDs
>>>>>>>>>>>> (which come for free with Quarkus) instead for the
>>> Correlation
>>>>> ID
>>>>>>> unless a
>>>>>>>>>>>> valid header is present, which would take over as the
>>>>> Correlation
>>>>>>> ID
>>>>>>>>>>>> instead.
>>>>>>>>>>>>>
>>>>>>>>>>>>> —
>>>>>>>>>>>>>
>>>>>>>>>>>>> To answer Dmitri’s question re Polaris Events: The
>>> intended
>>>>> use
>>>>>>> case is
>>>>>>>>>>>> to provide some sort of correlation between events that
>>> have
>>>>>>> occurred as
>>>>>>>>>>>> part of the same request. For example, if a user makes an
>>>>>>> CommitTransaction
>>>>>>>>>>>> request, it would be helpful to see all UpdateTable calls
>>> that
>>>>>>> were made as
>>>>>>>>>>>> part of that one user request.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
>>>>>>> dimas@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just to confirm: if / when we restore that, Polaris
>> will
>>> NOT
>>>>>>> generate
>>>>>>>>>>>> new
>>>>>>>>>>>>>> IDs in case the header is not present in the request,
>>>>> correct?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I believe x-request-id can co-exist with OTel.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What about adding request IDs to events [1][2]? What's
>>> the
>>>>>>> intended use
>>>>>>>>>>>>>> case for that? Could you share some context here too?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event persistence
>>> as
>>>>>>> "beta" in
>>>>>>>>>>>>>> 1.2.0... This discussion adds another point towards
>>> that, I
>>>>>>> think.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%252523L97%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Pioys4ROm8vYM_mdx4ygH&source=gmail-imap&ust=1762362311000000&usg=AOvVaw1ZMsu0wq2eWDF4OVIWaywJ
>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%252523L66%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw12-7e3ahm2sLSkLSNqTecm%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1PGW6uFwtUN8F1dOlJwrpm&source=gmail-imap&ust=1762362311000000&usg=AOvVaw385xLbQwQJRS5fmXlOPG8k
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [2877]
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2877%252523discussion_r2456300613%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H&source=gmail-imap&ust=1762362311000000&usg=AOvVaw1uAmRmbITVSySHFKIYJQ-f
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey Dmitri
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The generating a request id is new code that was added
>>>>> after the
>>>>>>>>>>>> original
>>>>>>>>>>>>>>> x-request-id support. You can see the state from ~1
>> year
>>>>> ago, we
>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>> request_id as the header we used for the MDC -
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%252523L415-L416%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw34DksOw7DSHJu8PfQQ5bT6&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2UoTrfPw_IJnFET_H3G_-M
>>>>>>>>>>>>>>> . At some point, it was changed to be configurable,
>>> then the
>>>>>>>>>>>>>>> ContextResolverFilter filter was refactored/eliminated
>>> and
>>>>> the
>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but lost some
>>> of
>>>>> its
>>>>>>> original
>>>>>>>>>>>>>>> functionality.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My personal opinion is that restoring support for the
>>>>>>> x-request-id
>>>>>>>>>>>> header
>>>>>>>>>>>>>>> is something that we should do, but if the header
>> isn't
>>>>> present,
>>>>>>>>>>>> falling
>>>>>>>>>>>>>>> back on simply using OTel trace ids is good enough
>>> (better,
>>>>>>> even) than
>>>>>>>>>>>>>>> generating another random request id.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for the info!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes sense to
>> me.
>>>>>>> However, I
>>>>>>>>>>>>>>> wonder:
>>>>>>>>>>>>>>>> why would Polaris need to generate a new request ID
>>> inside
>>>>> its
>>>>>>>>>>>> code?..
>>>>>>>>>>>>>>>> and return it in response headers?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How important is it to propagate this ID to Polaris
>>> Events?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm just trying to understand the full context :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think the original intention for this requestId
>>> field
>>>>> was to
>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>> request propagation from load balancers, like Envoy
>> (
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw3LNqtilZ2OoJfE7yEwraZa&source=gmail-imap&ust=1762362311000000&usg=AOvVaw0Rnn1QlkkrAmt4j8rz0gNl
>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask me why the
>>>>> default
>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was originally a
>>> custom
>>>>>>> thing,
>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> we changed to integrate with existing conventions.
>>>>>>> Unfortunately,
>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>> through the code, I think that the actual functional
>>>>> plumbing
>>>>>>> has
>>>>>>>>>>>> been
>>>>>>>>>>>>>>>> lost
>>>>>>>>>>>>>>>>> in the course of multiple refactors around the call
>>>>> context
>>>>>>> and
>>>>>>>>>>>>>>>> resolver. I
>>>>>>>>>>>>>>>>> don't see references to that property or the
>>> underlying
>>>>>>> header.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id header feels
>>> like
>>>>>>> something
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> should definitely keep, especially when Polaris is
>> one
>>>>>>> service in a
>>>>>>>>>>>>>>> mesh
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> services that maybe don't have OTel integration.
>> I'm a
>>>>> fan of
>>>>>>> the
>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>> standard, but it's not entirely ubiquitous and there
>>> are
>>>>> many
>>>>>>>>>>>>>>> middleware
>>>>>>>>>>>>>>>>> layers that know how to forward on the x-request-id
>>>>> header.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
>>>>> snazy@snazy.de>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability with the
>>> existing
>>>>>>> de-facto
>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users to
>> integrate
>>>>> into
>>>>>>> their
>>>>>>>>>>>>>>>>>> observability platforms.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
>> Bourlatchkov <
>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Alex and All,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I certainly support the idea of following OTel
>>>>> standards for
>>>>>>>>>>>>>>>> achieving
>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests and/or events.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> As to what form the correlation data should take,
>> I
>>>>> believe
>>>>>>> it is
>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context" represents.
>>> So, I
>>>>>>> believe it
>>>>>>>>>>>>>>>> makes
>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard context
>>>>> propagators
>>>>>>> at the
>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
>>> information,
>>>>> then
>>>>>>>>>>>>>>> returning
>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>> other "correlation" data in the response is
>>> redundant, I
>>>>>>> think.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If the incoming request does not have OTel context
>>> info,
>>>>>>> what is
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
>>> "correlation
>>>>> ID"?
>>>>>>> How is
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> envisioned to be used?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If the intention is to correlate a Polaris
>> response
>>>>>>> (operation)
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> events
>>>>>>>>>>>>>>>>>>> that might have resulted from its execution, I
>>> believe a
>>>>>>> more
>>>>>>>>>>>>>>> robust
>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel trace info
>>>>>>> (starting a new
>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> necessary) into event data. Then, Polaris can also
>>>>> return
>>>>>>> the
>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>> context
>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a bit awkward
>>> from
>>>>> the
>>>>>>> OTel
>>>>>>>>>>>>>>>>>>> perspective, but might be an option for supporting
>>>>> custom
>>>>>>>>>>>>>>>> correlators.
>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The header
>> name
>>>>> could be
>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to extract
>> the
>>>>> trace
>>>>>>> ID from
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> response and from events and find related data.
>>>>> Granted, it
>>>>>>> will
>>>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to decode
>> trace
>>> IDs
>>>>>>> from the
>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>> context, but the format is standard and not
>>> complex. The
>>>>>>> benefit
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily integrate
>>> with
>>>>>>>>>>>>>>> OTel-compatible
>>>>>>>>>>>>>>>>>>> observability platforms regardless of whether any
>>>>> particular
>>>>>>>>>>>>>>>> deployment
>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
>>>>>>>>>>>> adutra@apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of "request ID",
>> but
>>> its
>>>>>>> purpose is
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
>>> observability
>>>>>>> feature to
>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR aims to
>>> rename it
>>>>> to
>>>>>>>>>>>>>>>>> "correlation
>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry standards
>>> [1].
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light overlaps
>> with
>>>>> core
>>>>>>>>>>>>>>> telemetry
>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in
>>>>> Polaris,
>>>>>>> each
>>>>>>>>>>>>>>>>>>>> request already has a trace ID and span ID,
>> making
>>> a
>>>>>>> separate
>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and span ID in
>>>>> Polaris
>>>>>>> events,
>>>>>>>>>>>>>>>>>>>> rather than the generated correlation ID, would
>>>>>>> significantly
>>>>>>>>>>>>>>>>> simplify
>>>>>>>>>>>>>>>>>>>> correlation of events with other traces.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following changes:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span
>>> ID as
>>>>> the
>>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>> ID for the request, instead of generating a
>> random
>>>>>>> correlation
>>>>>>>>>>>>>>> ID.
>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation
>>> ID
>>>>>>> header is
>>>>>>>>>>>>>>>>> present
>>>>>>>>>>>>>>>>>>>> in the request, use it.
>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions is met,
>>> generate
>>>>> a
>>>>>>> random
>>>>>>>>>>>>>>>>>>>> correlation ID.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best approach
>> when a
>>>>>>> correlation
>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>>> header is present in the request. However, I
>>> believe it
>>>>>>> would be
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is enabled, as
>>> OTel
>>>>>>> offers a
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server trace
>>> propagation,
>>>>>>> e.g. W3C
>>>>>>>>>>>>>>>>> Trace
>>>>>>>>>>>>>>>>>>>> Context propagation [2].
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2757%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Oe-25vtt4gLVSMFMtSVNg&source=gmail-imap&ust=1762362311000000&usg=AOvVaw0oKBg5hY3-80AeZXaXcgmo
>>>>>>>>>>>>>>>>>>>> [2]:
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1TRdkzABc_7U-_KZ1MZ59v&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2dn5YK73cKTciJwSnNt8pd
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi Alex,
There has been no change of mind - my position still remains the same as it has from the start of this thread. Quoting from the email you linked to:
> As a result, I’m a +1 on Michael’s opinion: we should remove the Request/Correlation ID generation and always use the OTel trace/span IDs (which come for free with Quarkus) instead for the Correlation ID unless a valid header is present, which would take over as the Correlation ID instead.
My opinion on not generating a Request/“Correlation" ID relies on the implementation detail/assumption that Quarkus provides an OTel Trace/span ID - and that we should use that instead. Manual generation of Request IDs should only be an option if we cannot use the OTel Trace/Span data. My reading was Michael has the same opinion in the quote below:
> My personal opinion is that restoring support for the x-request-id header
is something that we should do, but if the header isn't present, falling
back on simply using OTel trace ids is good enough (better, even) than
generating another random request id.
I apologize if the “Correlation ID” terminology in the thread above is what was confusing about it, but I am still confused why we are introducing such a topic (question asked in later emails in this thread).
I empathize that this current thread was not going anywhere - but clarifying questions I’m asking regarding the changes being proposed (and their implications) are also not being fully answered so it is hard for me to change my position. I see the new forked threads and I will respond to them early next week - thanks for creating those.
Have a great weekend!
Best,
Adnan Hemani
> On Oct 31, 2025, at 5:13 AM, Alexandre Dutra <ad...@apache.org> wrote:
>
> Hi Adnan,
>
> Let's please avoid revisiting previously agreed-upon topics. We had
> general agreement that generating request IDs is unnecessary. Mike's
> opinion [1] supports this, and even you agreed [2]. Therefore, I'm
> surprised by your sudden change of mind.
>
> With that, I believe this discussion is now going in circles. Let's
> split it into 2 new threads to address the remaining open issues
> *separately*:
>
> 1) Whether the request ID sent by the client should be echoed in the response.
> 2) How to persist request IDs and OTel traces.
>
> Thanks,
> Alex
>
> [1] https://www.google.com/url?q=https://lists.apache.org/thread/zkmj8slsl3xyd0kzjdvnjqhh4rz0grlw&source=gmail-imap&ust=1762517650000000&usg=AOvVaw17DyvK0iympMP3p_O6-Jmg
> [2] https://www.google.com/url?q=https://lists.apache.org/thread/gvk05h7tnv4b513gr2bmyobqzmm2sos2&source=gmail-imap&ust=1762517650000000&usg=AOvVaw2mfPZiYZHLGun5skcghZRX
>
> On Fri, Oct 31, 2025 at 1:35 AM Adnan Hemani
> <ad...@snowflake.com.invalid> wrote:
>>
>> Hi Dmitri,
>>
>> Thanks for this - I am understanding what you are saying directly, but I’m not sure I am understanding the implications of those statements. Can you please help clarify the following?
>>
>> Re: separation of “observability/debuggability” vs “functional” use cases:
>> I agree that the request ID is not a hard requirement for the correctness of the server or client, in general. Maybe there is a use case that exists, but I am not aware of that either. So what I’m gathering from this is that there is not really a “functional” use case here. If you’re referring to the ability of a upstream system (such as any load balancer) to mark a request ID and have Polaris respect that as “functional” - I’m not sure it’s really all that different than an “observability” use case either. I’m not convinced either way - but if you can expand further on this, it may help me see your point.
>>
>> Re: propagating request IDs upwards
>> I don’t find this awkward, TBH. This is how most web systems I’ve ever worked with work. The ability to request a specific Request ID tend to be optional, but receiving back a Request ID (regardless of whether it was generated for you, or you had requested it and it was respected) through a header is quite standard. I don’t see how Polaris is any different - happy to hear why this would be the case.
>>
>> But even in the case that a client is trying to relate requests for a certain “unit of work” on their end, they have the ability to do so by using the `X-Request-Id` request header and Polaris should respect that value and resend the same request ID back as a response header. This actually makes things even clearer for the client to confirm that their Request ID was indeed respected.
>>
>> Thoughts?
>>
>> Best,
>> Adnan Hemani
>>
>>
>>
>>> On Oct 30, 2025, at 3:53 PM, Dmitri Bourlatchkov <di...@apache.org> wrote:
>>>
>>> Hi Adnan,
>>>
>>> Re: request ID headers in responses (point 4 in your email):
>>>
>>> I believe we need to distinguish observability / debuggability use cases
>>> from "functional" use of request IDs. By functional I mean clients that
>>> actually rely on the request ID value in order to perform their job
>>> properly.
>>>
>>> I do believe we're discussing the first use case in this thread. Meaning
>>> that request IDs are auxiliary pieces of information that are not critical
>>> for correctness of operation in the larger context. However, please correct
>>> me if I'm wrong here.
>>>
>>> That said, for observability / debugging use cases one normally establishes
>>> the request ID as early as possible and then propagates it to lower-level
>>> systems. Propagating request IDs upward is awkward because a client making
>>> multiple calls to Polaris as part of the same logical request on its side
>>> will end up with different polaris request IDs, so linking them to the
>>> client-side "unit of work" will require extra machinery.
>>>
>>> WDYT?
>>>
>>> Thanks,
>>> Dmitri.
>>>
>>> On Thu, Oct 30, 2025 at 5:44 PM Adnan Hemani
>>> <ad...@snowflake.com.invalid> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> Unfortunately, I don’t agree on most of the discussion past Alex’s
>>>> re-summarization. I do, however, agree on the requirements stated by Alex
>>>> in that email.
>>>>
>>>> Here’s what I believe we should be doing in summary:
>>>> 1. Keep the existing `request_id` column.
>>>> * To be fair, I don’t fully understand the reason why we are
>>>> trying to change a ubiquitous concept such as `request_id` to something
>>>> named as `correlation_id`. Are we trying to state explicitly that the
>>>> correlation ID and request ID should/are not be the same? If someone can
>>>> please explain this part, that would be helpful.
>>>> 2. In the `request_id` field, use the `X-Request-Id` header value if it
>>>> exists - as that is explicitly what the client is asking for and will mark
>>>> as the request ID in their logs.
>>>> 2a. Otherwise, use the OTel Trace ID as the request ID. This
>>>> should always exist, as it is provided by Quarkus on a per-request basis.
>>>> 2b. If, in the future, we are not provided an OTel Trace ID (for
>>>> whatever reason), one should be generated.
>>>> 4. Keep the `request_id` as part of the response headers.
>>>> * This allows clients to note down what the request ID that was
>>>> used and/or generated. Without it, how will clients who don’t provide a
>>>> `X-Request-Id` header be able to communicate which request they are
>>>> referring to if they need to debug something?
>>>> 5. I’m unsure on indexing over request_id or keeping the request_id_type.
>>>> I don’t see these as a hard use case today, but we can always add it as an
>>>> improvement in the future.
>>>>
>>>> To put this all another way (perhaps more philosophically, but the way I
>>>> am thinking about things in general):
>>>> * There should be exactly 1 request ID that is used when a client makes a
>>>> request to Polaris. That request ID can be dictated by the client (which I
>>>> still find a bit odd when looking at web server architecture in general,
>>>> but I understand the use case we have for it) or generated by the server.
>>>> * That request ID should then be used in any persisting anything that
>>>> involve request identification (in this particular case, for Events, but
>>>> not limited to if there’s something else in the future.)
>>>> * That request ID should be made aware of to the client, so it knows how
>>>> to identify the request in the case of future debugging.
>>>>
>>>> Please let me know your thoughts.
>>>>
>>>> Best,
>>>> Adnan Hemani
>>>>
>>>>
>>>>> On Oct 30, 2025, at 10:26 AM, Dmitri Bourlatchkov <di...@apache.org>
>>>> wrote:
>>>>>
>>>>> Hi Alex,
>>>>>
>>>>> SGTM (both not using request ID _response_ headers and deferring type
>>>> info
>>>>> in correlation ID values).
>>>>>
>>>>> Cheers,
>>>>> Dmitri.
>>>>>
>>>>> On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org>
>>>> wrote:
>>>>>
>>>>>> Hi Dmitri,
>>>>>>
>>>>>>> I believe we still need to clarify / confirm whether returning any
>>>>>> request / correlation ID in _response_ headers is required.
>>>>>>
>>>>>> Indeed, that specific item was still pending. FYI I'm going to
>>>>>> *remove* the echoing of the request ID in the response headers, unless
>>>>>> someone objects.
>>>>>>
>>>>>>> it could be useful to embed some type data into the value to help with
>>>>>> interpretation
>>>>>>
>>>>>> I'm not sure we need this at this moment. I'd suggest that we leave
>>>>>> this for further improvements in order to not over-specify the
>>>>>> requirements. Given that the events feature is beta, this shouldn't be
>>>>>> a problem.
>>>>>>
>>>>>> Thanks,
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> Thanks for the nice summary, Alex!
>>>>>>>
>>>>>>> To All:
>>>>>>>
>>>>>>> I'm not sure an INDEX on correlation_id is that useful... Polaris
>>>> itself
>>>>>>> does not use that index, right? Do we expect end users to query events
>>>>>>> directly from the database? This will work only with RDBMS, but
>>>> probably
>>>>>>> not with the proposed NoSQL persistence.
>>>>>>>
>>>>>>> I believe we still need to clarify / confirm whether returning any
>>>>>> request
>>>>>>> / correlation ID in _response_ headers is required.
>>>>>>>
>>>>>>> From my POV, I do not see a use case for it, but I wonder what other
>>>>>> people
>>>>>>> think / need.
>>>>>>>
>>>>>>> As for persisting correlation ID type, I agree that we do not need a
>>>>>>> separate column for that. Still, it could be useful to embed some type
>>>>>> data
>>>>>>> into the value to help with interpretation, specifically for telemetry
>>>>>> use
>>>>>>> cases (both x-request-id and OTel). For example, we could
>>>>>>> persist "traceparent:00-12345-...". Then, the reader would be able to
>>>>>>> identify this as information following the W3C spec [1], parse it and
>>>>>> link
>>>>>>> to other traces (if desired). That processing is optional and at the
>>>>>>> discretion of the event data reader, but making the value no-so-opaque
>>>>>>> upfront can enable a lot of interesting options for users without much
>>>>>>> extra overhead in Polaris. WDYT?
>>>>>>>
>>>>>>> [1]
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/2021/REC-trace-context-1-20211123/%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw1K633xlm1Dj0NjcNcpd41A%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw3FJYjDvXW1ifT-dq78bLOy&source=gmail-imap&ust=1762517650000000&usg=AOvVaw0VAEGmE5ZuVFt-2WAIrIDI
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dmitri.
>>>>>>>
>>>>>>> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>>> we would only persist one or the other, not both.
>>>>>>>>
>>>>>>>> My apologies, I confess that this wasn't my understanding of the
>>>>>> desired
>>>>>>>> design.
>>>>>>>>
>>>>>>>> Let's try to gather the requirements again:
>>>>>>>>
>>>>>>>> 1. Only one correlation ID is enough
>>>>>>>> 2. The correlation ID is an opaque string
>>>>>>>> 3. The main use case is to find events with matching correlation IDs
>>>>>>>> 4. The only query pattern is by exact match
>>>>>>>>
>>>>>>>> Regarding *where* to store the correlation ID, and expanding on what
>>>>>>>> Dmitri said: the requirement to find matching records by correlation
>>>>>>>> ID clearly suggests a separate column, and with an INDEX on it.
>>>>>>>> Storing it in additional_properties will be inefficient.
>>>>>>>>
>>>>>>>> If we all agree on the above, here is what I propose:
>>>>>>>>
>>>>>>>> 1. Rename the existing column request_id to correlation_id;
>>>>>>>> 2. Populate from OTel context if available; the exact syntax is
>>>>>>>> implementation-specific;
>>>>>>>> 3. Otherwise, populate from X-Request-Id header if available;
>>>>>>>> 4. Otherwise, don't populate.
>>>>>>>> 5. Create an INDEX on correlation_id.
>>>>>>>>
>>>>>>>> You'll notice that we don't store the ID type, but that doesn't seem
>>>>>>>> necessary.
>>>>>>>>
>>>>>>>> What do you all think?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <dimas@apache.org
>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Yufei,
>>>>>>>>>
>>>>>>>>> [...] tying a BeforeUpdateTable event to the corresponding
>>>>>>>> AfterUpdateTable
>>>>>>>>> event.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It looks like this calls for a strong and always present ID in
>>>>>>>> events.This
>>>>>>>>> ID is obviously distincts from OTel context information and should be
>>>>>>>>> persisted separately, IMHO.
>>>>>>>>>
>>>>>>>>> I believe there's still an open question whether the request ID in
>>>>>> the
>>>>>>>>> events should be populated from the x-request-id in REST API headers.
>>>>>>>>>
>>>>>>>>> Yufei: Is that required in your use cases?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Dmitri.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I think the first question is whether any downstream use cases
>>>>>> need to
>>>>>>>>>> interpret the semantics of a request ID or OTel context. As I
>>>>>>>> mentioned,
>>>>>>>>>> the original purpose of a request ID was event correlation, e.g.,
>>>>>>>> tying a
>>>>>>>>>> BeforeUpdateTable event to the corresponding AfterUpdateTable
>>>>>> event. If
>>>>>>>>>> that’s the only requirement, an opaque correlation ID is good
>>>>>> enough.
>>>>>>>> We
>>>>>>>>>> could change the field name(request-id) if needed.
>>>>>>>>>>
>>>>>>>>>> If we do have use cases that depend on the meaning of the request
>>>>>> ID or
>>>>>>>>>> OTel context, we can store them in additional_properties (
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%252523L134-L134%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw3kHvvqJhJTJi61gR6c6CFy%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw0kUL91EJEo0g8DuALtlFaF&source=gmail-imap&ust=1762517650000000&usg=AOvVaw0R3wH0wfTXp1dyoMybn4lj
>>>>>>>>>> ).
>>>>>>>>>> This is more flexible because it allows us to:
>>>>>>>>>> 1. Record a key that indicates the format (e.g., x-request-id vs.
>>>>>>>>>> otel-context).
>>>>>>>>>> 2. Preserve both when Polaris receives both headers.
>>>>>>>>>> 3. Add future tracing fields without a schema change.
>>>>>>>>>>
>>>>>>>>>> Yufei
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
>>>>>> dimas@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> From my POV the use cases for x-request-id and OTel are slightly
>>>>>>>>>> different.
>>>>>>>>>>> OTel is an observability use case where cross-linking of various
>>>>>>>> pieces
>>>>>>>>>> of
>>>>>>>>>>> data is desired but not required in all cases (data loss is
>>>>>>>> acceptable).
>>>>>>>>>>> The x-request-id use case appears to have some very specific
>>>>>> custom
>>>>>>>>>>> applications without any obvious relationship to OTel.
>>>>>>>>>>>
>>>>>>>>>>> Given the current state of this discussion, I think it makes
>>>>>> sense to
>>>>>>>>>>> handle and persist them separately.
>>>>>>>>>>>
>>>>>>>>>>> However, if we intend x-request-id to act as an observability aid
>>>>>>>>>>> alternative to OTel, then it might be preferable to have a config
>>>>>>>> flag
>>>>>>>>>>> indicating which one of them should be added to persistent event
>>>>>>>> data. In
>>>>>>>>>>> that case we should reuse the same column for them (and possibly
>>>>>> add
>>>>>>>> a
>>>>>>>>>>> "type" prefix, e.g. "x-request-id:ABCDEF" or
>>>>>>>> "traceparent:00-12345...")
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Dmitri.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
>>>>>> adutra@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Yufei,
>>>>>>>>>>>>
>>>>>>>>>>>>> I’d suggest persisting a single opaque correlation ID instead
>>>>>>>>>>>>
>>>>>>>>>>>> The problem I see with this is that readers cannot know the
>>>>>> syntax
>>>>>>>> of
>>>>>>>>>>>> the opaque id, and will have to "probe" using some heuristics,
>>>>>>>> e.g. if
>>>>>>>>>>>> the id matches the OTel trace context syntax, asume it's OTel,
>>>>>>>>>>>> otherwise... assume it's something else.
>>>>>>>>>>>>
>>>>>>>>>>>> That's not unfeasible, but I wonder if it isn't better to store
>>>>>>>> the 2
>>>>>>>>>>>> ids separately.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Alex
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Alex for raising this discussion. Thanks everyone for
>>>>>>>> chiming
>>>>>>>>>>> in.
>>>>>>>>>>>>> The plan looks good to me overall.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Update PolarisEvent, expose request ID if available, expose
>>>>>>>> OTel
>>>>>>>>>>>> context
>>>>>>>>>>>>> if available.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2914%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw33p8lOR4oYEJqcbo4Sm74f%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw1aWh6q_9-TX1Ao82MTOmiZ&source=gmail-imap&ust=1762517650000000&usg=AOvVaw2UKrJb4GOxfIc07D-y9z2Q
>>>> persists both
>>>>>>>> request id
>>>>>>>>>>> and
>>>>>>>>>>>>> otel context.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do we actually need to persist both?
>>>>>>>>>>>>> From the original discussion, the intent of a request ID was
>>>>>> to
>>>>>>>>>>> correlate
>>>>>>>>>>>>> multiple events generated by the same request, for example,
>>>>>>>> linking a
>>>>>>>>>>>>> BeforeUpdateTable and an AfterUpdateTable event. In that
>>>>>> sense,
>>>>>>>> what
>>>>>>>>>> we
>>>>>>>>>>>>> really need is a single opaque identifier that ties related
>>>>>>>> events
>>>>>>>>>>>>> together, regardless of whether it comes from a request ID
>>>>>> or an
>>>>>>>> OTel
>>>>>>>>>>>>> context.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we persist both fields (requestId and otelContext), it can
>>>>>>>> lead to
>>>>>>>>>>>>> consumer confusion, since downstream systems would need to
>>>>>> decide
>>>>>>>>>> which
>>>>>>>>>>>> one
>>>>>>>>>>>>> to rely on, potentially causing inconsistent event
>>>>>> correlation.
>>>>>>>>>> Unless
>>>>>>>>>>>>> there are concrete use cases where event consumers need
>>>>>> both, I’d
>>>>>>>>>>> suggest
>>>>>>>>>>>>> persisting a single opaque correlation ID instead. This
>>>>>> keeps the
>>>>>>>>>>> schema
>>>>>>>>>>>>> simpler and ensures consistent event linking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yufei
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
>>>>>>>> adutra@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Here is the PR:
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2914%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw33p8lOR4oYEJqcbo4Sm74f%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw1aWh6q_9-TX1Ao82MTOmiZ&source=gmail-imap&ust=1762517650000000&usg=AOvVaw2UKrJb4GOxfIc07D-y9z2Q
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
>>>>>>>>>>> jb@nanthrax.net>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah it looks good to me and I agree we have a consensus.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
>>>>>>>> adutra@apache.org
>>>>>>>>>>>
>>>>>>>>>>> a
>>>>>>>>>>>>>> écrit :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for the good discussion so far. I think we have
>>>>>> a
>>>>>>>> broad
>>>>>>>>>>>>>> consensus.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Unless anybody disagrees I am going to move forward
>>>>>> with
>>>>>>>> the
>>>>>>>>>> the
>>>>>>>>>>>>>>>> following revised plan, incorporating the latest
>>>>>> feedback:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Restore the original functionality for request IDs,
>>>>>>>> change
>>>>>>>>>> the
>>>>>>>>>>>>>>>> default header name back to x-request-id
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 3. Update PolarisEvent, expose request ID if available,
>>>>>>>> expose
>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>> context if available.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 4. Update events table SQL schema: insert request ID if
>>>>>>>>>>> available,
>>>>>>>>>>>>>>>> insert OTel context if available.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
>>>>>>>>>>>> jb@nanthrax.net
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think it's better to use w3c_trace_context (better
>>>>>>>> than two
>>>>>>>>>>>>>> fields).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regarding x-request-id, I think it's for a different
>>>>>>>> purpose,
>>>>>>>>>>>>>>>>> specifically for persistence. Personally, I think we
>>>>>> can
>>>>>>>>>>> achieve
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> same with span and events.
>>>>>>>>>>>>>>>>> If we want to "distinguish" the two layers, it makes
>>>>>>>> sense to
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> x-request-id (not sure it's super helpful). Else, I
>>>>>>>> think we
>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> achieve the same with span/event.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Just my $0.01
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
>>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested
>>>>>>>> changes as
>>>>>>>>>>>> well.
>>>>>>>>>>>>>> Great
>>>>>>>>>>>>>>>> find, Dmitri!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I’m still debating with myself on whether we need
>>>>>> to
>>>>>>>> store
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> `x-request-id` field as part of the Events persistence.
>>>>>>>> Can we
>>>>>>>>>>>> think
>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>> good use case where this is more helpful to the user
>>>>>> than
>>>>>>>> the
>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>> Trace/Span IDs? I am making the assumption here that
>>>>>> those
>>>>>>>> are
>>>>>>>>>>>> still
>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>> returned back to the client.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
>>>>>>>>>>>> adutra@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Dmitri,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yes, I think your suggestion to use just one
>>>>>> field
>>>>>>>>>>>>>> w3c_trace_context
>>>>>>>>>>>>>>>>>>> makes more sense than two fields (span_id and
>>>>>>>> trace_id).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> With that, I also think that we are slowly
>>>>>> drifting
>>>>>>>> into
>>>>>>>>>>>>>>>>>>> implementation considerations; let's get
>>>>>> consensus
>>>>>>>> on the
>>>>>>>>>>>> general
>>>>>>>>>>>>>>>>>>> design first, and we can certainly fine-tune the
>>>>>>>> actual
>>>>>>>>>>> Java
>>>>>>>>>>>>>> methods
>>>>>>>>>>>>>>>>>>> and SQL table columns in the future PR. WDYT?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri
>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>> dimas@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Many thanks for the background info, Adnan!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1 to action items proposed by Alex!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Re: (3) Can we abstract request/trace info into
>>>>>> a
>>>>>>>>>> separate
>>>>>>>>>>>>>> object
>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>>>> exposing those accessors on the Event class
>>>>>>>> directly?
>>>>>>>>>> OTel
>>>>>>>>>>>>>> defines
>>>>>>>>>>>>>>>>>>>> trace/span concepts, but in request ID is a bit
>>>>>>>> foreign
>>>>>>>>>> to
>>>>>>>>>>>> OTel.
>>>>>>>>>>>>>>>> Having
>>>>>>>>>>>>>>>>>>>> tracing / request ID isolated in java could help
>>>>>>>> with
>>>>>>>>>>>>>> maintaining
>>>>>>>>>>>>>>>> it and
>>>>>>>>>>>>>>>>>>>> potentially supporting other (custom) tracing
>>>>>>>> methods.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Re: (4) I'd like to propose storing OTel
>>>>>> correlation
>>>>>>>>>> data
>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>> form of a
>>>>>>>>>>>>>>>>>>>> standard context propagation string, e.g. W3C
>>>>>>>>>>> trace-context
>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>> (same value
>>>>>>>>>>>>>>>>>>>> as its HTTP header), so the column could be
>>>>>> called
>>>>>>>>>>>>>>>> w3c_trace_context or
>>>>>>>>>>>>>>>>>>>> simply trace_context.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Open question: do we need to write a separate,
>>>>>>>>>> individual
>>>>>>>>>>>> trace
>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>> field in
>>>>>>>>>>>>>>>>>>>> SQL? I suppose it is not very useful since
>>>>>>>> correlating
>>>>>>>>>> it
>>>>>>>>>>> to
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>> data already requires understanding OTel context
>>>>>>>>>>> propagation
>>>>>>>>>>>>>> and a
>>>>>>>>>>>>>>>> query
>>>>>>>>>>>>>>>>>>>> against trace_context can still be made using
>>>>>>>>>>>> string-matching
>>>>>>>>>>>>>>>> clauses. We
>>>>>>>>>>>>>>>>>>>> could probably (additionally) store it in the
>>>>>>>> request_id
>>>>>>>>>>>> column
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> Polaris-specific request ID header is not set.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As for span ID, I do not really see a use case
>>>>>> for
>>>>>>>>>>>> persisting it
>>>>>>>>>>>>>>>>>>>> individually. It is very specific to OTel trace
>>>>>> data
>>>>>>>>>>>>>> construction.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Actually, using the W3C trace context [1]
>>>>>> encoding
>>>>>>>>>>> probably
>>>>>>>>>>>>>> makes
>>>>>>>>>>>>>>>> sense in
>>>>>>>>>>>>>>>>>>>> the java event representation too. Interested
>>>>>>>> callers
>>>>>>>>>> can
>>>>>>>>>>>> easily
>>>>>>>>>>>>>>>> decode
>>>>>>>>>>>>>>>>>>>> this information since the format is
>>>>>> well-defined.
>>>>>>>> As a
>>>>>>>>>>> side
>>>>>>>>>>>>>>>> benefit, this
>>>>>>>>>>>>>>>>>>>> opens opportunities for downstream event
>>>>>> consumers
>>>>>>>> to
>>>>>>>>>>>> connect
>>>>>>>>>>>>>>>> (propagate
>>>>>>>>>>>>>>>>>>>> context) to traces that produced events based
>>>>>> on the
>>>>>>>>>> event
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>> itself,
>>>>>>>>>>>>>>>>>>>> without relying on the intermediate frameworks.
>>>>>>>> This may
>>>>>>>>>>> be
>>>>>>>>>>>>>>>> desirable since
>>>>>>>>>>>>>>>>>>>> the current Polaris event persistence impl.
>>>>>> writes
>>>>>>>>>> events
>>>>>>>>>>> in
>>>>>>>>>>>>>>>> batches, so
>>>>>>>>>>>>>>>>>>>> the association to individual requests that
>>>>>> produced
>>>>>>>>>> those
>>>>>>>>>>>>>> events
>>>>>>>>>>>>>>>> is lost.
>>>>>>>>>>>>>>>>>>>> Whether to perform this kind of context
>>>>>> propagation
>>>>>>>> will
>>>>>>>>>>> be
>>>>>>>>>>>> at
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> discretion of the event consumer, of course
>>>>>>>> (outside of
>>>>>>>>>>>> Polaris
>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.w3.org/TR/trace-context/%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw2fn0lRsTx-f9r4PCz8wmJK%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw0JHBUVrCE7tookVBiFjNdP%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw2tTZYzZmafSmGBVhT4wv7Z&source=gmail-imap&ust=1762517650000000&usg=AOvVaw3Gh5Tu8EaPs91MWl2Y2YaM
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
>>>>>> <
>>>>>>>>>>>>>> adutra@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thank you for chiming in; the context around
>>>>>>>> request
>>>>>>>>>> IDs
>>>>>>>>>>>> is now
>>>>>>>>>>>>>>>> clear.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Trying to summarize this thread into actionable
>>>>>>>> items,
>>>>>>>>>>>> here's
>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> propose:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 1. Restore the original functionality for
>>>>>> request
>>>>>>>> IDs.
>>>>>>>>>>>>>>>>>>>>> - Change the default header name back to
>>>>>>>>>> x-request-id
>>>>>>>>>>>>>> (despite
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> x- prefix being deprecated), but keep it
>>>>>>>> configurable
>>>>>>>>>> as
>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related
>>>>>>>> functionality.
>>>>>>>>>>>>>>>>>>>>> - Do not generate a request ID if the REST
>>>>>>>> client
>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>> provide one.
>>>>>>>>>>>>>>>>>>>>> 3. Update PolarisEvent:
>>>>>>>>>>>>>>>>>>>>> - Expose new requestId(), traceId(),
>>>>>> spanId()
>>>>>>>>>> methods,
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>> nullable.
>>>>>>>>>>>>>>>>>>>>> - This would align with the emerging
>>>>>> consensus
>>>>>>>>>> around
>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>> contextual information in PolarisEvent [1].
>>>>>>>>>>>>>>>>>>>>> 4. Update events table SQL schema:
>>>>>>>>>>>>>>>>>>>>> - Insert the client-provided request ID
>>>>>> into the
>>>>>>>>>>>> request_id
>>>>>>>>>>>>>>>>>>>>> column; otherwise, insert null.
>>>>>>>>>>>>>>>>>>>>> - Add two new nullable columns, trace_id and
>>>>>>>>>> span_id,
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> populate
>>>>>>>>>>>>>>>>>>>>> them if OTel is enabled.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> From our discussions, I think it's important
>>>>>> to not
>>>>>>>>>>>> conflate
>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>> tracing fields with Envoy tracing fields,
>>>>>> which is
>>>>>>>> why
>>>>>>>>>> I
>>>>>>>>>>>>>> suggest we
>>>>>>>>>>>>>>>>>>>>> use separate fields / columns for them.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Would the above plan work for everyone?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw02wPvb0qxRzYAKEP0h8l9T%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw3AIjY12CwTp7ItIvzTOIIN%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw0O6N5bjiJtEUkTDiZL-DlH&source=gmail-imap&ust=1762517650000000&usg=AOvVaw0yIPH0owkzMa4SXyDjw9vq
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
>>>>>>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks to Alex for starting this thread -
>>>>>> because
>>>>>>>> of
>>>>>>>>>>>> this, I’m
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>> coming to the realization that OTel Trace and
>>>>>> Span
>>>>>>>> IDs
>>>>>>>>>>> are
>>>>>>>>>>>>>> coming
>>>>>>>>>>>>>>>> built-in
>>>>>>>>>>>>>>>>>>>>> with Quarkus and my previous work to generate a
>>>>>>>>>>>>>>>> Request/Correlation ID is
>>>>>>>>>>>>>>>>>>>>> likely not needed as a result. My original
>>>>>>>> motivation
>>>>>>>>>> for
>>>>>>>>>>>>>>>> generation of a
>>>>>>>>>>>>>>>>>>>>> Request/Correlation ID was to ensure that any
>>>>>>>> client
>>>>>>>>>> can
>>>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>> identify
>>>>>>>>>>>>>>>>>>>>> a request made to Polaris, which would be
>>>>>>>> especially
>>>>>>>>>>>> useful for
>>>>>>>>>>>>>>>> debugging
>>>>>>>>>>>>>>>>>>>>> failing requests or identifying call patterns.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we
>>>>>>>> should
>>>>>>>>>>>> remove
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> Request/Correlation ID generation and always
>>>>>> use
>>>>>>>> the
>>>>>>>>>> OTel
>>>>>>>>>>>>>>>> trace/span IDs
>>>>>>>>>>>>>>>>>>>>> (which come for free with Quarkus) instead for
>>>>>> the
>>>>>>>>>>>> Correlation
>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>> unless a
>>>>>>>>>>>>>>>>>>>>> valid header is present, which would take over
>>>>>> as
>>>>>>>> the
>>>>>>>>>>>>>> Correlation
>>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>>>> instead.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> To answer Dmitri’s question re Polaris
>>>>>> Events: The
>>>>>>>>>>>> intended
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>> case is
>>>>>>>>>>>>>>>>>>>>> to provide some sort of correlation between
>>>>>> events
>>>>>>>> that
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> occurred as
>>>>>>>>>>>>>>>>>>>>> part of the same request. For example, if a
>>>>>> user
>>>>>>>> makes
>>>>>>>>>> an
>>>>>>>>>>>>>>>> CommitTransaction
>>>>>>>>>>>>>>>>>>>>> request, it would be helpful to see all
>>>>>> UpdateTable
>>>>>>>>>> calls
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> were made as
>>>>>>>>>>>>>>>>>>>>> part of that one user request.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
>>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Just to confirm: if / when we restore that,
>>>>>>>> Polaris
>>>>>>>>>>> will
>>>>>>>>>>>> NOT
>>>>>>>>>>>>>>>> generate
>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>> IDs in case the header is not present in the
>>>>>>>> request,
>>>>>>>>>>>>>> correct?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I believe x-request-id can co-exist with
>>>>>> OTel.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What about adding request IDs to events
>>>>>> [1][2]?
>>>>>>>>>> What's
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> intended use
>>>>>>>>>>>>>>>>>>>>>>> case for that? Could you share some context
>>>>>> here
>>>>>>>> too?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event
>>>>>>>>>> persistence
>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> "beta" in
>>>>>>>>>>>>>>>>>>>>>>> 1.2.0... This discussion adds another point
>>>>>>>> towards
>>>>>>>>>>>> that, I
>>>>>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2525252523L97%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw1Pioys4ROm8vYM_mdx4ygH%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw39NasMg4Tgyrfe5Bj_U4xY%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw0sDeqLAsJrlT0vl5OSGLZC&source=gmail-imap&ust=1762517650000000&usg=AOvVaw3C_kJ4xU0hNzM9PQJlKNyO
>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2525252523L66%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw12-7e3ahm2sLSkLSNqTecm%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw1PGW6uFwtUN8F1dOlJwrpm%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw3y_SObfpEJH_K_8RD9lVTL%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw0Iwahb3E3xhtLPtdPmqs1i&source=gmail-imap&ust=1762517650000000&usg=AOvVaw0EeQ_7N63a0H-KsuGHAP0Z
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> [2877]
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://github.com/apache/polaris/pull/2877%2525252523discussion_r2456300613%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw2_e3B3qU8kJxdMmQx-q6bZ%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw3BMSIO2LpE1BaeIgK-c9nY&source=gmail-imap&ust=1762517650000000&usg=AOvVaw3NVz_UIs7y_tI9qTHf2tpb
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
>>>>>> Collado <
>>>>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hey Dmitri
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The generating a request id is new code
>>>>>> that was
>>>>>>>>>> added
>>>>>>>>>>>>>> after the
>>>>>>>>>>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>>>>>> x-request-id support. You can see the state
>>>>>>>> from ~1
>>>>>>>>>>> year
>>>>>>>>>>>>>> ago, we
>>>>>>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>>>>>>>>> request_id as the header we used for the
>>>>>> MDC -
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2525252523L415-L416%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw34DksOw7DSHJu8PfQQ5bT6%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw1g09FwhrTquJs9utV9CAX9%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw0dwBsX019LeuexMDdck0dM&source=gmail-imap&ust=1762517650000000&usg=AOvVaw3Ouplyu_xqEXLemFnMaIA6
>>>>>>>>>>>>>>>>>>>>>>>> . At some point, it was changed to be
>>>>>>>> configurable,
>>>>>>>>>>>> then the
>>>>>>>>>>>>>>>>>>>>>>>> ContextResolverFilter filter was
>>>>>>>>>> refactored/eliminated
>>>>>>>>>>>> and
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but
>>>>>>>> lost
>>>>>>>>>> some
>>>>>>>>>>>> of
>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>>>>>> functionality.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> My personal opinion is that restoring
>>>>>> support
>>>>>>>> for
>>>>>>>>>> the
>>>>>>>>>>>>>>>> x-request-id
>>>>>>>>>>>>>>>>>>>>> header
>>>>>>>>>>>>>>>>>>>>>>>> is something that we should do, but if the
>>>>>>>> header
>>>>>>>>>>> isn't
>>>>>>>>>>>>>> present,
>>>>>>>>>>>>>>>>>>>>> falling
>>>>>>>>>>>>>>>>>>>>>>>> back on simply using OTel trace ids is good
>>>>>>>> enough
>>>>>>>>>>>> (better,
>>>>>>>>>>>>>>>> even) than
>>>>>>>>>>>>>>>>>>>>>>>> generating another random request id.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
>>>>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the info!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes
>>>>>>>> sense to
>>>>>>>>>>> me.
>>>>>>>>>>>>>>>> However, I
>>>>>>>>>>>>>>>>>>>>>>>> wonder:
>>>>>>>>>>>>>>>>>>>>>>>>> why would Polaris need to generate a new
>>>>>>>> request ID
>>>>>>>>>>>> inside
>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>> code?..
>>>>>>>>>>>>>>>>>>>>>>>>> and return it in response headers?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> How important is it to propagate this ID to
>>>>>>>> Polaris
>>>>>>>>>>>> Events?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I'm just trying to understand the full
>>>>>> context
>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
>>>>>>>> Collado <
>>>>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I think the original intention for this
>>>>>>>> requestId
>>>>>>>>>>>> field
>>>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>> request propagation from load balancers,
>>>>>> like
>>>>>>>>>> Envoy
>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw3LNqtilZ2OoJfE7yEwraZa%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw2hTMXu2DU30Lg_5UBzsDJw%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw3EG5R-XtDVNQoOZTJYC4BU&source=gmail-imap&ust=1762517650000000&usg=AOvVaw30dDzIR8U8IZMrkZ2QTPP9
>>>>>>>>>>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask
>>>>>> me
>>>>>>>> why
>>>>>>>>>> the
>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was
>>>>>>>> originally
>>>>>>>>>> a
>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>> thing,
>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>> we changed to integrate with existing
>>>>>>>> conventions.
>>>>>>>>>>>>>>>> Unfortunately,
>>>>>>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>>>> through the code, I think that the actual
>>>>>>>>>> functional
>>>>>>>>>>>>>> plumbing
>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>> lost
>>>>>>>>>>>>>>>>>>>>>>>>>> in the course of multiple refactors
>>>>>> around the
>>>>>>>>>> call
>>>>>>>>>>>>>> context
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> resolver. I
>>>>>>>>>>>>>>>>>>>>>>>>>> don't see references to that property or
>>>>>> the
>>>>>>>>>>>> underlying
>>>>>>>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id
>>>>>> header
>>>>>>>>>> feels
>>>>>>>>>>>> like
>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>> should definitely keep, especially when
>>>>>>>> Polaris is
>>>>>>>>>>> one
>>>>>>>>>>>>>>>> service in a
>>>>>>>>>>>>>>>>>>>>>>>> mesh
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> services that maybe don't have OTel
>>>>>>>> integration.
>>>>>>>>>>> I'm a
>>>>>>>>>>>>>> fan of
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>>>> standard, but it's not entirely
>>>>>> ubiquitous and
>>>>>>>>>> there
>>>>>>>>>>>> are
>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>>>>>>>> middleware
>>>>>>>>>>>>>>>>>>>>>>>>>> layers that know how to forward on the
>>>>>>>>>> x-request-id
>>>>>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
>>>>>> Stupp <
>>>>>>>>>>>>>> snazy@snazy.de>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability
>>>>>> with
>>>>>>>> the
>>>>>>>>>>>> existing
>>>>>>>>>>>>>>>> de-facto
>>>>>>>>>>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users
>>>>>> to
>>>>>>>>>>> integrate
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>> observability platforms.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
>>>>>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Alex and All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I certainly support the idea of
>>>>>> following
>>>>>>>> OTel
>>>>>>>>>>>>>> standards for
>>>>>>>>>>>>>>>>>>>>>>>>> achieving
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests
>>>>>> and/or
>>>>>>>>>> events.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As to what form the correlation data
>>>>>> should
>>>>>>>>>> take,
>>>>>>>>>>> I
>>>>>>>>>>>>>> believe
>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context"
>>>>>>>> represents.
>>>>>>>>>>>> So, I
>>>>>>>>>>>>>>>> believe it
>>>>>>>>>>>>>>>>>>>>>>>>> makes
>>>>>>>>>>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard
>>>>>>>> context
>>>>>>>>>>>>>> propagators
>>>>>>>>>>>>>>>> at the
>>>>>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
>>>>>>>>>>>> information,
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>> returning
>>>>>>>>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>>>>>> other "correlation" data in the
>>>>>> response is
>>>>>>>>>>>> redundant, I
>>>>>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request does not have
>>>>>> OTel
>>>>>>>>>> context
>>>>>>>>>>>> info,
>>>>>>>>>>>>>>>> what is
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
>>>>>>>>>>>> "correlation
>>>>>>>>>>>>>> ID"?
>>>>>>>>>>>>>>>> How is
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioned to be used?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the intention is to correlate a
>>>>>> Polaris
>>>>>>>>>>> response
>>>>>>>>>>>>>>>> (operation)
>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>> events
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that might have resulted from its
>>>>>>>> execution, I
>>>>>>>>>>>> believe a
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>> robust
>>>>>>>>>>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel
>>>>>>>> trace
>>>>>>>>>> info
>>>>>>>>>>>>>>>> (starting a new
>>>>>>>>>>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary) into event data. Then,
>>>>>> Polaris
>>>>>>>> can
>>>>>>>>>> also
>>>>>>>>>>>>>> return
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>>>>>> context
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a
>>>>>> bit
>>>>>>>>>> awkward
>>>>>>>>>>>> from
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>>>>>> perspective, but might be an option for
>>>>>>>>>> supporting
>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>>>>>>>>>> correlators.
>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The
>>>>>>>> header
>>>>>>>>>>> name
>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace
>>>>>> Context.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to
>>>>>>>> extract
>>>>>>>>>>> the
>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>> ID from
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> response and from events and find
>>>>>> related
>>>>>>>> data.
>>>>>>>>>>>>>> Granted, it
>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to
>>>>>>>> decode
>>>>>>>>>>> trace
>>>>>>>>>>>> IDs
>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>>>>>> context, but the format is standard and
>>>>>> not
>>>>>>>>>>>> complex. The
>>>>>>>>>>>>>>>> benefit
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily
>>>>>>>> integrate
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> OTel-compatible
>>>>>>>>>>>>>>>>>>>>>>>>>>>> observability platforms regardless of
>>>>>>>> whether
>>>>>>>>>> any
>>>>>>>>>>>>>> particular
>>>>>>>>>>>>>>>>>>>>>>>>> deployment
>>>>>>>>>>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
>>>>>> Alexandre
>>>>>>>> Dutra
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>> adutra@apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of
>>>>>> "request
>>>>>>>> ID",
>>>>>>>>>>> but
>>>>>>>>>>>> its
>>>>>>>>>>>>>>>> purpose is
>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
>>>>>>>>>>>> observability
>>>>>>>>>>>>>>>> feature to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR
>>>>>> aims
>>>>>>>> to
>>>>>>>>>>>> rename it
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> "correlation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry
>>>>>>>>>> standards
>>>>>>>>>>>> [1].
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light
>>>>>>>> overlaps
>>>>>>>>>>> with
>>>>>>>>>>>>>> core
>>>>>>>>>>>>>>>>>>>>>>>> telemetry
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is
>>>>>>>> enabled
>>>>>>>>>> in
>>>>>>>>>>>>>> Polaris,
>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> request already has a trace ID and
>>>>>> span ID,
>>>>>>>>>>> making
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> separate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and
>>>>>> span
>>>>>>>> ID
>>>>>>>>>> in
>>>>>>>>>>>>>> Polaris
>>>>>>>>>>>>>>>> events,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the generated correlation
>>>>>> ID,
>>>>>>>> would
>>>>>>>>>>>>>>>> significantly
>>>>>>>>>>>>>>>>>>>>>>>>>> simplify
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation of events with other
>>>>>> traces.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following
>>>>>> changes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace
>>>>>> ID and
>>>>>>>>>> span
>>>>>>>>>>>> ID as
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ID for the request, instead of
>>>>>> generating a
>>>>>>>>>>> random
>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>>>> ID.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
>>>>>>>>>> correlation
>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>> header is
>>>>>>>>>>>>>>>>>>>>>>>>>> present
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the request, use it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions
>>>>>> is
>>>>>>>> met,
>>>>>>>>>>>> generate
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> random
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best
>>>>>>>> approach
>>>>>>>>>>> when a
>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> header is present in the request.
>>>>>> However,
>>>>>>>> I
>>>>>>>>>>>> believe it
>>>>>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is
>>>>>>>> enabled, as
>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>> offers a
>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server
>>>>>> trace
>>>>>>>>>>>> propagation,
>>>>>>>>>>>>>>>> e.g. W3C
>>>>>>>>>>>>>>>>>>>>>>>>>> Trace
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Context propagation [2].
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://github.com/apache/polaris/pull/2757%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw1Oe-25vtt4gLVSMFMtSVNg%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw2ohYKOQ8XU5-GBlDo6zYGC%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw38kU5KZM7IlDD0Y7UPVYn5&source=gmail-imap&ust=1762517650000000&usg=AOvVaw1LIST0aNzl2NN6XuOGvMdz
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.google.com/url?q%2525253Dhttps://www.w3.org/TR/trace-context%25252526source%2525253Dgmail-imap%25252526ust%2525253D1761851831000000%25252526usg%2525253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%252526source%25253Dgmail-imap%252526ust%25253D1761927167000000%252526usg%25253DAOvVaw1TRdkzABc_7U-_KZ1MZ59v%2526source%253Dgmail-imap%2526ust%253D1762450034000000%2526usg%253DAOvVaw0cUBLtXCkRoH5pL57naUhY%26source%3Dgmail-imap%26ust%3D1762469626000000%26usg%3DAOvVaw3UFfRQLLzFZTMF1vmrxROq&source=gmail-imap&ust=1762517650000000&usg=AOvVaw0AKkO9Mcxw4cwE4BReQE6J
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>>
>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi Adnan,
Let's please avoid revisiting previously agreed-upon topics. We had
general agreement that generating request IDs is unnecessary. Mike's
opinion [1] supports this, and even you agreed [2]. Therefore, I'm
surprised by your sudden change of mind.
With that, I believe this discussion is now going in circles. Let's
split it into 2 new threads to address the remaining open issues
*separately*:
1) Whether the request ID sent by the client should be echoed in the response.
2) How to persist request IDs and OTel traces.
Thanks,
Alex
[1] https://lists.apache.org/thread/zkmj8slsl3xyd0kzjdvnjqhh4rz0grlw
[2] https://lists.apache.org/thread/gvk05h7tnv4b513gr2bmyobqzmm2sos2
On Fri, Oct 31, 2025 at 1:35 AM Adnan Hemani
<ad...@snowflake.com.invalid> wrote:
>
> Hi Dmitri,
>
> Thanks for this - I am understanding what you are saying directly, but I’m not sure I am understanding the implications of those statements. Can you please help clarify the following?
>
> Re: separation of “observability/debuggability” vs “functional” use cases:
> I agree that the request ID is not a hard requirement for the correctness of the server or client, in general. Maybe there is a use case that exists, but I am not aware of that either. So what I’m gathering from this is that there is not really a “functional” use case here. If you’re referring to the ability of a upstream system (such as any load balancer) to mark a request ID and have Polaris respect that as “functional” - I’m not sure it’s really all that different than an “observability” use case either. I’m not convinced either way - but if you can expand further on this, it may help me see your point.
>
> Re: propagating request IDs upwards
> I don’t find this awkward, TBH. This is how most web systems I’ve ever worked with work. The ability to request a specific Request ID tend to be optional, but receiving back a Request ID (regardless of whether it was generated for you, or you had requested it and it was respected) through a header is quite standard. I don’t see how Polaris is any different - happy to hear why this would be the case.
>
> But even in the case that a client is trying to relate requests for a certain “unit of work” on their end, they have the ability to do so by using the `X-Request-Id` request header and Polaris should respect that value and resend the same request ID back as a response header. This actually makes things even clearer for the client to confirm that their Request ID was indeed respected.
>
> Thoughts?
>
> Best,
> Adnan Hemani
>
>
>
> > On Oct 30, 2025, at 3:53 PM, Dmitri Bourlatchkov <di...@apache.org> wrote:
> >
> > Hi Adnan,
> >
> > Re: request ID headers in responses (point 4 in your email):
> >
> > I believe we need to distinguish observability / debuggability use cases
> > from "functional" use of request IDs. By functional I mean clients that
> > actually rely on the request ID value in order to perform their job
> > properly.
> >
> > I do believe we're discussing the first use case in this thread. Meaning
> > that request IDs are auxiliary pieces of information that are not critical
> > for correctness of operation in the larger context. However, please correct
> > me if I'm wrong here.
> >
> > That said, for observability / debugging use cases one normally establishes
> > the request ID as early as possible and then propagates it to lower-level
> > systems. Propagating request IDs upward is awkward because a client making
> > multiple calls to Polaris as part of the same logical request on its side
> > will end up with different polaris request IDs, so linking them to the
> > client-side "unit of work" will require extra machinery.
> >
> > WDYT?
> >
> > Thanks,
> > Dmitri.
> >
> > On Thu, Oct 30, 2025 at 5:44 PM Adnan Hemani
> > <ad...@snowflake.com.invalid> wrote:
> >
> >> Hi folks,
> >>
> >> Unfortunately, I don’t agree on most of the discussion past Alex’s
> >> re-summarization. I do, however, agree on the requirements stated by Alex
> >> in that email.
> >>
> >> Here’s what I believe we should be doing in summary:
> >> 1. Keep the existing `request_id` column.
> >> * To be fair, I don’t fully understand the reason why we are
> >> trying to change a ubiquitous concept such as `request_id` to something
> >> named as `correlation_id`. Are we trying to state explicitly that the
> >> correlation ID and request ID should/are not be the same? If someone can
> >> please explain this part, that would be helpful.
> >> 2. In the `request_id` field, use the `X-Request-Id` header value if it
> >> exists - as that is explicitly what the client is asking for and will mark
> >> as the request ID in their logs.
> >> 2a. Otherwise, use the OTel Trace ID as the request ID. This
> >> should always exist, as it is provided by Quarkus on a per-request basis.
> >> 2b. If, in the future, we are not provided an OTel Trace ID (for
> >> whatever reason), one should be generated.
> >> 4. Keep the `request_id` as part of the response headers.
> >> * This allows clients to note down what the request ID that was
> >> used and/or generated. Without it, how will clients who don’t provide a
> >> `X-Request-Id` header be able to communicate which request they are
> >> referring to if they need to debug something?
> >> 5. I’m unsure on indexing over request_id or keeping the request_id_type.
> >> I don’t see these as a hard use case today, but we can always add it as an
> >> improvement in the future.
> >>
> >> To put this all another way (perhaps more philosophically, but the way I
> >> am thinking about things in general):
> >> * There should be exactly 1 request ID that is used when a client makes a
> >> request to Polaris. That request ID can be dictated by the client (which I
> >> still find a bit odd when looking at web server architecture in general,
> >> but I understand the use case we have for it) or generated by the server.
> >> * That request ID should then be used in any persisting anything that
> >> involve request identification (in this particular case, for Events, but
> >> not limited to if there’s something else in the future.)
> >> * That request ID should be made aware of to the client, so it knows how
> >> to identify the request in the case of future debugging.
> >>
> >> Please let me know your thoughts.
> >>
> >> Best,
> >> Adnan Hemani
> >>
> >>
> >>> On Oct 30, 2025, at 10:26 AM, Dmitri Bourlatchkov <di...@apache.org>
> >> wrote:
> >>>
> >>> Hi Alex,
> >>>
> >>> SGTM (both not using request ID _response_ headers and deferring type
> >> info
> >>> in correlation ID values).
> >>>
> >>> Cheers,
> >>> Dmitri.
> >>>
> >>> On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org>
> >> wrote:
> >>>
> >>>> Hi Dmitri,
> >>>>
> >>>>> I believe we still need to clarify / confirm whether returning any
> >>>> request / correlation ID in _response_ headers is required.
> >>>>
> >>>> Indeed, that specific item was still pending. FYI I'm going to
> >>>> *remove* the echoing of the request ID in the response headers, unless
> >>>> someone objects.
> >>>>
> >>>>> it could be useful to embed some type data into the value to help with
> >>>> interpretation
> >>>>
> >>>> I'm not sure we need this at this moment. I'd suggest that we leave
> >>>> this for further improvements in order to not over-specify the
> >>>> requirements. Given that the events feature is beta, this shouldn't be
> >>>> a problem.
> >>>>
> >>>> Thanks,
> >>>> Alex
> >>>>
> >>>>
> >>>> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
> >>>> wrote:
> >>>>>
> >>>>> Thanks for the nice summary, Alex!
> >>>>>
> >>>>> To All:
> >>>>>
> >>>>> I'm not sure an INDEX on correlation_id is that useful... Polaris
> >> itself
> >>>>> does not use that index, right? Do we expect end users to query events
> >>>>> directly from the database? This will work only with RDBMS, but
> >> probably
> >>>>> not with the proposed NoSQL persistence.
> >>>>>
> >>>>> I believe we still need to clarify / confirm whether returning any
> >>>> request
> >>>>> / correlation ID in _response_ headers is required.
> >>>>>
> >>>>> From my POV, I do not see a use case for it, but I wonder what other
> >>>> people
> >>>>> think / need.
> >>>>>
> >>>>> As for persisting correlation ID type, I agree that we do not need a
> >>>>> separate column for that. Still, it could be useful to embed some type
> >>>> data
> >>>>> into the value to help with interpretation, specifically for telemetry
> >>>> use
> >>>>> cases (both x-request-id and OTel). For example, we could
> >>>>> persist "traceparent:00-12345-...". Then, the reader would be able to
> >>>>> identify this as information following the W3C spec [1], parse it and
> >>>> link
> >>>>> to other traces (if desired). That processing is optional and at the
> >>>>> discretion of the event data reader, but making the value no-so-opaque
> >>>>> upfront can enable a lot of interesting options for users without much
> >>>>> extra overhead in Polaris. WDYT?
> >>>>>
> >>>>> [1]
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/2021/REC-trace-context-1-20211123/%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw1K633xlm1Dj0NjcNcpd41A&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3FJYjDvXW1ifT-dq78bLOy
> >>>>>
> >>>>> Thanks,
> >>>>> Dmitri.
> >>>>>
> >>>>> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
> >>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>>> we would only persist one or the other, not both.
> >>>>>>
> >>>>>> My apologies, I confess that this wasn't my understanding of the
> >>>> desired
> >>>>>> design.
> >>>>>>
> >>>>>> Let's try to gather the requirements again:
> >>>>>>
> >>>>>> 1. Only one correlation ID is enough
> >>>>>> 2. The correlation ID is an opaque string
> >>>>>> 3. The main use case is to find events with matching correlation IDs
> >>>>>> 4. The only query pattern is by exact match
> >>>>>>
> >>>>>> Regarding *where* to store the correlation ID, and expanding on what
> >>>>>> Dmitri said: the requirement to find matching records by correlation
> >>>>>> ID clearly suggests a separate column, and with an INDEX on it.
> >>>>>> Storing it in additional_properties will be inefficient.
> >>>>>>
> >>>>>> If we all agree on the above, here is what I propose:
> >>>>>>
> >>>>>> 1. Rename the existing column request_id to correlation_id;
> >>>>>> 2. Populate from OTel context if available; the exact syntax is
> >>>>>> implementation-specific;
> >>>>>> 3. Otherwise, populate from X-Request-Id header if available;
> >>>>>> 4. Otherwise, don't populate.
> >>>>>> 5. Create an INDEX on correlation_id.
> >>>>>>
> >>>>>> You'll notice that we don't store the ID type, but that doesn't seem
> >>>>>> necessary.
> >>>>>>
> >>>>>> What do you all think?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Alex
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <dimas@apache.org
> >>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi Yufei,
> >>>>>>>
> >>>>>>> [...] tying a BeforeUpdateTable event to the corresponding
> >>>>>> AfterUpdateTable
> >>>>>>> event.
> >>>>>>>
> >>>>>>>
> >>>>>>> It looks like this calls for a strong and always present ID in
> >>>>>> events.This
> >>>>>>> ID is obviously distincts from OTel context information and should be
> >>>>>>> persisted separately, IMHO.
> >>>>>>>
> >>>>>>> I believe there's still an open question whether the request ID in
> >>>> the
> >>>>>>> events should be populated from the x-request-id in REST API headers.
> >>>>>>>
> >>>>>>> Yufei: Is that required in your use cases?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dmitri.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> I think the first question is whether any downstream use cases
> >>>> need to
> >>>>>>>> interpret the semantics of a request ID or OTel context. As I
> >>>>>> mentioned,
> >>>>>>>> the original purpose of a request ID was event correlation, e.g.,
> >>>>>> tying a
> >>>>>>>> BeforeUpdateTable event to the corresponding AfterUpdateTable
> >>>> event. If
> >>>>>>>> that’s the only requirement, an opaque correlation ID is good
> >>>> enough.
> >>>>>> We
> >>>>>>>> could change the field name(request-id) if needed.
> >>>>>>>>
> >>>>>>>> If we do have use cases that depend on the meaning of the request
> >>>> ID or
> >>>>>>>> OTel context, we can store them in additional_properties (
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%2523L134-L134%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3kHvvqJhJTJi61gR6c6CFy&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0kUL91EJEo0g8DuALtlFaF
> >>>>>>>> ).
> >>>>>>>> This is more flexible because it allows us to:
> >>>>>>>> 1. Record a key that indicates the format (e.g., x-request-id vs.
> >>>>>>>> otel-context).
> >>>>>>>> 2. Preserve both when Polaris receives both headers.
> >>>>>>>> 3. Add future tracing fields without a schema change.
> >>>>>>>>
> >>>>>>>> Yufei
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
> >>>> dimas@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi All,
> >>>>>>>>>
> >>>>>>>>> From my POV the use cases for x-request-id and OTel are slightly
> >>>>>>>> different.
> >>>>>>>>> OTel is an observability use case where cross-linking of various
> >>>>>> pieces
> >>>>>>>> of
> >>>>>>>>> data is desired but not required in all cases (data loss is
> >>>>>> acceptable).
> >>>>>>>>> The x-request-id use case appears to have some very specific
> >>>> custom
> >>>>>>>>> applications without any obvious relationship to OTel.
> >>>>>>>>>
> >>>>>>>>> Given the current state of this discussion, I think it makes
> >>>> sense to
> >>>>>>>>> handle and persist them separately.
> >>>>>>>>>
> >>>>>>>>> However, if we intend x-request-id to act as an observability aid
> >>>>>>>>> alternative to OTel, then it might be preferable to have a config
> >>>>>> flag
> >>>>>>>>> indicating which one of them should be added to persistent event
> >>>>>> data. In
> >>>>>>>>> that case we should reuse the same column for them (and possibly
> >>>> add
> >>>>>> a
> >>>>>>>>> "type" prefix, e.g. "x-request-id:ABCDEF" or
> >>>>>> "traceparent:00-12345...")
> >>>>>>>>>
> >>>>>>>>> WDYT?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Dmitri.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
> >>>> adutra@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Yufei,
> >>>>>>>>>>
> >>>>>>>>>>> I’d suggest persisting a single opaque correlation ID instead
> >>>>>>>>>>
> >>>>>>>>>> The problem I see with this is that readers cannot know the
> >>>> syntax
> >>>>>> of
> >>>>>>>>>> the opaque id, and will have to "probe" using some heuristics,
> >>>>>> e.g. if
> >>>>>>>>>> the id matches the OTel trace context syntax, asume it's OTel,
> >>>>>>>>>> otherwise... assume it's something else.
> >>>>>>>>>>
> >>>>>>>>>> That's not unfeasible, but I wonder if it isn't better to store
> >>>>>> the 2
> >>>>>>>>>> ids separately.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Alex
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
> >>>>>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks Alex for raising this discussion. Thanks everyone for
> >>>>>> chiming
> >>>>>>>>> in.
> >>>>>>>>>>> The plan looks good to me overall.
> >>>>>>>>>>>
> >>>>>>>>>>>> Update PolarisEvent, expose request ID if available, expose
> >>>>>> OTel
> >>>>>>>>>> context
> >>>>>>>>>>> if available.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2914%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw33p8lOR4oYEJqcbo4Sm74f&source=gmail-imap&ust=1762469626000000&usg=AOvVaw1aWh6q_9-TX1Ao82MTOmiZ
> >> persists both
> >>>>>> request id
> >>>>>>>>> and
> >>>>>>>>>>> otel context.
> >>>>>>>>>>>
> >>>>>>>>>>> Do we actually need to persist both?
> >>>>>>>>>>> From the original discussion, the intent of a request ID was
> >>>> to
> >>>>>>>>> correlate
> >>>>>>>>>>> multiple events generated by the same request, for example,
> >>>>>> linking a
> >>>>>>>>>>> BeforeUpdateTable and an AfterUpdateTable event. In that
> >>>> sense,
> >>>>>> what
> >>>>>>>> we
> >>>>>>>>>>> really need is a single opaque identifier that ties related
> >>>>>> events
> >>>>>>>>>>> together, regardless of whether it comes from a request ID
> >>>> or an
> >>>>>> OTel
> >>>>>>>>>>> context.
> >>>>>>>>>>>
> >>>>>>>>>>> If we persist both fields (requestId and otelContext), it can
> >>>>>> lead to
> >>>>>>>>>>> consumer confusion, since downstream systems would need to
> >>>> decide
> >>>>>>>> which
> >>>>>>>>>> one
> >>>>>>>>>>> to rely on, potentially causing inconsistent event
> >>>> correlation.
> >>>>>>>> Unless
> >>>>>>>>>>> there are concrete use cases where event consumers need
> >>>> both, I’d
> >>>>>>>>> suggest
> >>>>>>>>>>> persisting a single opaque correlation ID instead. This
> >>>> keeps the
> >>>>>>>>> schema
> >>>>>>>>>>> simpler and ensures consistent event linking.
> >>>>>>>>>>>
> >>>>>>>>>>> Yufei
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
> >>>>>> adutra@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Here is the PR:
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2914%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw33p8lOR4oYEJqcbo4Sm74f&source=gmail-imap&ust=1762469626000000&usg=AOvVaw1aWh6q_9-TX1Ao82MTOmiZ
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Alex
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> >>>>>>>>> jb@nanthrax.net>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yeah it looks good to me and I agree we have a consensus.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks !
> >>>>>>>>>>>>> Regards
> >>>>>>>>>>>>> JB
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
> >>>>>> adutra@apache.org
> >>>>>>>>>
> >>>>>>>>> a
> >>>>>>>>>>>> écrit :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the good discussion so far. I think we have
> >>>> a
> >>>>>> broad
> >>>>>>>>>>>> consensus.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Unless anybody disagrees I am going to move forward
> >>>> with
> >>>>>> the
> >>>>>>>> the
> >>>>>>>>>>>>>> following revised plan, incorporating the latest
> >>>> feedback:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 1. Restore the original functionality for request IDs,
> >>>>>> change
> >>>>>>>> the
> >>>>>>>>>>>>>> default header name back to x-request-id
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 3. Update PolarisEvent, expose request ID if available,
> >>>>>> expose
> >>>>>>>>> OTel
> >>>>>>>>>>>>>> context if available.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 4. Update events table SQL schema: insert request ID if
> >>>>>>>>> available,
> >>>>>>>>>>>>>> insert OTel context if available.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> >>>>>>>>>> jb@nanthrax.net
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think it's better to use w3c_trace_context (better
> >>>>>> than two
> >>>>>>>>>>>> fields).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Regarding x-request-id, I think it's for a different
> >>>>>> purpose,
> >>>>>>>>>>>>>>> specifically for persistence. Personally, I think we
> >>>> can
> >>>>>>>>> achieve
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> same with span and events.
> >>>>>>>>>>>>>>> If we want to "distinguish" the two layers, it makes
> >>>>>> sense to
> >>>>>>>>>> have
> >>>>>>>>>>>>>>> x-request-id (not sure it's super helpful). Else, I
> >>>>>> think we
> >>>>>>>>> can
> >>>>>>>>>>>>>>> achieve the same with span/event.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Just my $0.01
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Regards
> >>>>>>>>>>>>>>> JB
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> >>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested
> >>>>>> changes as
> >>>>>>>>>> well.
> >>>>>>>>>>>> Great
> >>>>>>>>>>>>>> find, Dmitri!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I’m still debating with myself on whether we need
> >>>> to
> >>>>>> store
> >>>>>>>>> the
> >>>>>>>>>>>>>> `x-request-id` field as part of the Events persistence.
> >>>>>> Can we
> >>>>>>>>>> think
> >>>>>>>>>>>> of a
> >>>>>>>>>>>>>> good use case where this is more helpful to the user
> >>>> than
> >>>>>> the
> >>>>>>>>> OTel
> >>>>>>>>>>>>>> Trace/Span IDs? I am making the assumption here that
> >>>> those
> >>>>>> are
> >>>>>>>>>> still
> >>>>>>>>>>>> being
> >>>>>>>>>>>>>> returned back to the client.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>> Adnan Hemani
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> >>>>>>>>>> adutra@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi Dmitri,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yes, I think your suggestion to use just one
> >>>> field
> >>>>>>>>>>>> w3c_trace_context
> >>>>>>>>>>>>>>>>> makes more sense than two fields (span_id and
> >>>>>> trace_id).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> With that, I also think that we are slowly
> >>>> drifting
> >>>>>> into
> >>>>>>>>>>>>>>>>> implementation considerations; let's get
> >>>> consensus
> >>>>>> on the
> >>>>>>>>>> general
> >>>>>>>>>>>>>>>>> design first, and we can certainly fine-tune the
> >>>>>> actual
> >>>>>>>>> Java
> >>>>>>>>>>>> methods
> >>>>>>>>>>>>>>>>> and SQL table columns in the future PR. WDYT?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri
> >>>> Bourlatchkov <
> >>>>>>>>>>>>>> dimas@apache.org> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Many thanks for the background info, Adnan!
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> +1 to action items proposed by Alex!
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Re: (3) Can we abstract request/trace info into
> >>>> a
> >>>>>>>> separate
> >>>>>>>>>>>> object
> >>>>>>>>>>>>>> without
> >>>>>>>>>>>>>>>>>> exposing those accessors on the Event class
> >>>>>> directly?
> >>>>>>>> OTel
> >>>>>>>>>>>> defines
> >>>>>>>>>>>>>>>>>> trace/span concepts, but in request ID is a bit
> >>>>>> foreign
> >>>>>>>> to
> >>>>>>>>>> OTel.
> >>>>>>>>>>>>>> Having
> >>>>>>>>>>>>>>>>>> tracing / request ID isolated in java could help
> >>>>>> with
> >>>>>>>>>>>> maintaining
> >>>>>>>>>>>>>> it and
> >>>>>>>>>>>>>>>>>> potentially supporting other (custom) tracing
> >>>>>> methods.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Re: (4) I'd like to propose storing OTel
> >>>> correlation
> >>>>>>>> data
> >>>>>>>>>> in the
> >>>>>>>>>>>>>> form of a
> >>>>>>>>>>>>>>>>>> standard context propagation string, e.g. W3C
> >>>>>>>>> trace-context
> >>>>>>>>>> [1]
> >>>>>>>>>>>>>> (same value
> >>>>>>>>>>>>>>>>>> as its HTTP header), so the column could be
> >>>> called
> >>>>>>>>>>>>>> w3c_trace_context or
> >>>>>>>>>>>>>>>>>> simply trace_context.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Open question: do we need to write a separate,
> >>>>>>>> individual
> >>>>>>>>>> trace
> >>>>>>>>>>>> ID
> >>>>>>>>>>>>>> field in
> >>>>>>>>>>>>>>>>>> SQL? I suppose it is not very useful since
> >>>>>> correlating
> >>>>>>>> it
> >>>>>>>>> to
> >>>>>>>>>>>> other
> >>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>> data already requires understanding OTel context
> >>>>>>>>> propagation
> >>>>>>>>>>>> and a
> >>>>>>>>>>>>>> query
> >>>>>>>>>>>>>>>>>> against trace_context can still be made using
> >>>>>>>>>> string-matching
> >>>>>>>>>>>>>> clauses. We
> >>>>>>>>>>>>>>>>>> could probably (additionally) store it in the
> >>>>>> request_id
> >>>>>>>>>> column
> >>>>>>>>>>>> if
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> Polaris-specific request ID header is not set.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As for span ID, I do not really see a use case
> >>>> for
> >>>>>>>>>> persisting it
> >>>>>>>>>>>>>>>>>> individually. It is very specific to OTel trace
> >>>> data
> >>>>>>>>>>>> construction.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Actually, using the W3C trace context [1]
> >>>> encoding
> >>>>>>>>> probably
> >>>>>>>>>>>> makes
> >>>>>>>>>>>>>> sense in
> >>>>>>>>>>>>>>>>>> the java event representation too. Interested
> >>>>>> callers
> >>>>>>>> can
> >>>>>>>>>> easily
> >>>>>>>>>>>>>> decode
> >>>>>>>>>>>>>>>>>> this information since the format is
> >>>> well-defined.
> >>>>>> As a
> >>>>>>>>> side
> >>>>>>>>>>>>>> benefit, this
> >>>>>>>>>>>>>>>>>> opens opportunities for downstream event
> >>>> consumers
> >>>>>> to
> >>>>>>>>>> connect
> >>>>>>>>>>>>>> (propagate
> >>>>>>>>>>>>>>>>>> context) to traces that produced events based
> >>>> on the
> >>>>>>>> event
> >>>>>>>>>> data
> >>>>>>>>>>>>>> itself,
> >>>>>>>>>>>>>>>>>> without relying on the intermediate frameworks.
> >>>>>> This may
> >>>>>>>>> be
> >>>>>>>>>>>>>> desirable since
> >>>>>>>>>>>>>>>>>> the current Polaris event persistence impl.
> >>>> writes
> >>>>>>>> events
> >>>>>>>>> in
> >>>>>>>>>>>>>> batches, so
> >>>>>>>>>>>>>>>>>> the association to individual requests that
> >>>> produced
> >>>>>>>> those
> >>>>>>>>>>>> events
> >>>>>>>>>>>>>> is lost.
> >>>>>>>>>>>>>>>>>> Whether to perform this kind of context
> >>>> propagation
> >>>>>> will
> >>>>>>>>> be
> >>>>>>>>>> at
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> discretion of the event consumer, of course
> >>>>>> (outside of
> >>>>>>>>>> Polaris
> >>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context/%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw2fn0lRsTx-f9r4PCz8wmJK%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw0JHBUVrCE7tookVBiFjNdP&source=gmail-imap&ust=1762469626000000&usg=AOvVaw2tTZYzZmafSmGBVhT4wv7Z
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
> >>>> <
> >>>>>>>>>>>> adutra@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thank you for chiming in; the context around
> >>>>>> request
> >>>>>>>> IDs
> >>>>>>>>>> is now
> >>>>>>>>>>>>>> clear.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Trying to summarize this thread into actionable
> >>>>>> items,
> >>>>>>>>>> here's
> >>>>>>>>>>>> what
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> propose:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1. Restore the original functionality for
> >>>> request
> >>>>>> IDs.
> >>>>>>>>>>>>>>>>>>> - Change the default header name back to
> >>>>>>>> x-request-id
> >>>>>>>>>>>> (despite
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> x- prefix being deprecated), but keep it
> >>>>>> configurable
> >>>>>>>> as
> >>>>>>>>>> today.
> >>>>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related
> >>>>>> functionality.
> >>>>>>>>>>>>>>>>>>> - Do not generate a request ID if the REST
> >>>>>> client
> >>>>>>>>>> doesn't
> >>>>>>>>>>>>>> provide one.
> >>>>>>>>>>>>>>>>>>> 3. Update PolarisEvent:
> >>>>>>>>>>>>>>>>>>> - Expose new requestId(), traceId(),
> >>>> spanId()
> >>>>>>>> methods,
> >>>>>>>>>> all
> >>>>>>>>>>>>>> nullable.
> >>>>>>>>>>>>>>>>>>> - This would align with the emerging
> >>>> consensus
> >>>>>>>> around
> >>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>> contextual information in PolarisEvent [1].
> >>>>>>>>>>>>>>>>>>> 4. Update events table SQL schema:
> >>>>>>>>>>>>>>>>>>> - Insert the client-provided request ID
> >>>> into the
> >>>>>>>>>> request_id
> >>>>>>>>>>>>>>>>>>> column; otherwise, insert null.
> >>>>>>>>>>>>>>>>>>> - Add two new nullable columns, trace_id and
> >>>>>>>> span_id,
> >>>>>>>>>> and
> >>>>>>>>>>>>>> populate
> >>>>>>>>>>>>>>>>>>> them if OTel is enabled.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> From our discussions, I think it's important
> >>>> to not
> >>>>>>>>>> conflate
> >>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>> tracing fields with Envoy tracing fields,
> >>>> which is
> >>>>>> why
> >>>>>>>> I
> >>>>>>>>>>>> suggest we
> >>>>>>>>>>>>>>>>>>> use separate fields / columns for them.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Would the above plan work for everyone?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [1]:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw02wPvb0qxRzYAKEP0h8l9T%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3AIjY12CwTp7ItIvzTOIIN&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0O6N5bjiJtEUkTDiZL-DlH
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> >>>>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks to Alex for starting this thread -
> >>>> because
> >>>>>> of
> >>>>>>>>>> this, I’m
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> coming to the realization that OTel Trace and
> >>>> Span
> >>>>>> IDs
> >>>>>>>>> are
> >>>>>>>>>>>> coming
> >>>>>>>>>>>>>> built-in
> >>>>>>>>>>>>>>>>>>> with Quarkus and my previous work to generate a
> >>>>>>>>>>>>>> Request/Correlation ID is
> >>>>>>>>>>>>>>>>>>> likely not needed as a result. My original
> >>>>>> motivation
> >>>>>>>> for
> >>>>>>>>>>>>>> generation of a
> >>>>>>>>>>>>>>>>>>> Request/Correlation ID was to ensure that any
> >>>>>> client
> >>>>>>>> can
> >>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>> identify
> >>>>>>>>>>>>>>>>>>> a request made to Polaris, which would be
> >>>>>> especially
> >>>>>>>>>> useful for
> >>>>>>>>>>>>>> debugging
> >>>>>>>>>>>>>>>>>>> failing requests or identifying call patterns.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we
> >>>>>> should
> >>>>>>>>>> remove
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> Request/Correlation ID generation and always
> >>>> use
> >>>>>> the
> >>>>>>>> OTel
> >>>>>>>>>>>>>> trace/span IDs
> >>>>>>>>>>>>>>>>>>> (which come for free with Quarkus) instead for
> >>>> the
> >>>>>>>>>> Correlation
> >>>>>>>>>>>> ID
> >>>>>>>>>>>>>> unless a
> >>>>>>>>>>>>>>>>>>> valid header is present, which would take over
> >>>> as
> >>>>>> the
> >>>>>>>>>>>> Correlation
> >>>>>>>>>>>>>> ID
> >>>>>>>>>>>>>>>>>>> instead.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> To answer Dmitri’s question re Polaris
> >>>> Events: The
> >>>>>>>>>> intended
> >>>>>>>>>>>> use
> >>>>>>>>>>>>>> case is
> >>>>>>>>>>>>>>>>>>> to provide some sort of correlation between
> >>>> events
> >>>>>> that
> >>>>>>>>>> have
> >>>>>>>>>>>>>> occurred as
> >>>>>>>>>>>>>>>>>>> part of the same request. For example, if a
> >>>> user
> >>>>>> makes
> >>>>>>>> an
> >>>>>>>>>>>>>> CommitTransaction
> >>>>>>>>>>>>>>>>>>> request, it would be helpful to see all
> >>>> UpdateTable
> >>>>>>>> calls
> >>>>>>>>>> that
> >>>>>>>>>>>>>> were made as
> >>>>>>>>>>>>>>>>>>> part of that one user request.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>> Adnan Hemani
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
> >>>>>> Bourlatchkov <
> >>>>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi Michael,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Just to confirm: if / when we restore that,
> >>>>>> Polaris
> >>>>>>>>> will
> >>>>>>>>>> NOT
> >>>>>>>>>>>>>> generate
> >>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>> IDs in case the header is not present in the
> >>>>>> request,
> >>>>>>>>>>>> correct?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I believe x-request-id can co-exist with
> >>>> OTel.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What about adding request IDs to events
> >>>> [1][2]?
> >>>>>>>> What's
> >>>>>>>>>> the
> >>>>>>>>>>>>>> intended use
> >>>>>>>>>>>>>>>>>>>>> case for that? Could you share some context
> >>>> here
> >>>>>> too?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event
> >>>>>>>> persistence
> >>>>>>>>>> as
> >>>>>>>>>>>>>> "beta" in
> >>>>>>>>>>>>>>>>>>>>> 1.2.0... This discussion adds another point
> >>>>>> towards
> >>>>>>>>>> that, I
> >>>>>>>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%25252523L97%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1Pioys4ROm8vYM_mdx4ygH%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw39NasMg4Tgyrfe5Bj_U4xY&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0sDeqLAsJrlT0vl5OSGLZC
> >>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%25252523L66%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw12-7e3ahm2sLSkLSNqTecm%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1PGW6uFwtUN8F1dOlJwrpm%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3y_SObfpEJH_K_8RD9lVTL&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0Iwahb3E3xhtLPtdPmqs1i
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> [2877]
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/pull/2877%25252523discussion_r2456300613%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2_e3B3qU8kJxdMmQx-q6bZ&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3BMSIO2LpE1BaeIgK-c9nY
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
> >>>> Collado <
> >>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hey Dmitri
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The generating a request id is new code
> >>>> that was
> >>>>>>>> added
> >>>>>>>>>>>> after the
> >>>>>>>>>>>>>>>>>>> original
> >>>>>>>>>>>>>>>>>>>>>> x-request-id support. You can see the state
> >>>>>> from ~1
> >>>>>>>>> year
> >>>>>>>>>>>> ago, we
> >>>>>>>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>>>>>>>>> request_id as the header we used for the
> >>>> MDC -
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%25252523L415-L416%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw34DksOw7DSHJu8PfQQ5bT6%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw1g09FwhrTquJs9utV9CAX9&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0dwBsX019LeuexMDdck0dM
> >>>>>>>>>>>>>>>>>>>>>> . At some point, it was changed to be
> >>>>>> configurable,
> >>>>>>>>>> then the
> >>>>>>>>>>>>>>>>>>>>>> ContextResolverFilter filter was
> >>>>>>>> refactored/eliminated
> >>>>>>>>>> and
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but
> >>>>>> lost
> >>>>>>>> some
> >>>>>>>>>> of
> >>>>>>>>>>>> its
> >>>>>>>>>>>>>> original
> >>>>>>>>>>>>>>>>>>>>>> functionality.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> My personal opinion is that restoring
> >>>> support
> >>>>>> for
> >>>>>>>> the
> >>>>>>>>>>>>>> x-request-id
> >>>>>>>>>>>>>>>>>>> header
> >>>>>>>>>>>>>>>>>>>>>> is something that we should do, but if the
> >>>>>> header
> >>>>>>>>> isn't
> >>>>>>>>>>>> present,
> >>>>>>>>>>>>>>>>>>> falling
> >>>>>>>>>>>>>>>>>>>>>> back on simply using OTel trace ids is good
> >>>>>> enough
> >>>>>>>>>> (better,
> >>>>>>>>>>>>>> even) than
> >>>>>>>>>>>>>>>>>>>>>> generating another random request id.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> >>>>>>>> Bourlatchkov <
> >>>>>>>>>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Michael,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the info!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes
> >>>>>> sense to
> >>>>>>>>> me.
> >>>>>>>>>>>>>> However, I
> >>>>>>>>>>>>>>>>>>>>>> wonder:
> >>>>>>>>>>>>>>>>>>>>>>> why would Polaris need to generate a new
> >>>>>> request ID
> >>>>>>>>>> inside
> >>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> code?..
> >>>>>>>>>>>>>>>>>>>>>>> and return it in response headers?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> How important is it to propagate this ID to
> >>>>>> Polaris
> >>>>>>>>>> Events?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I'm just trying to understand the full
> >>>> context
> >>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
> >>>>>> Collado <
> >>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I think the original intention for this
> >>>>>> requestId
> >>>>>>>>>> field
> >>>>>>>>>>>> was to
> >>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> request propagation from load balancers,
> >>>> like
> >>>>>>>> Envoy
> >>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw3LNqtilZ2OoJfE7yEwraZa%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2hTMXu2DU30Lg_5UBzsDJw&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3EG5R-XtDVNQoOZTJYC4BU
> >>>>>>>>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask
> >>>> me
> >>>>>> why
> >>>>>>>> the
> >>>>>>>>>>>> default
> >>>>>>>>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was
> >>>>>> originally
> >>>>>>>> a
> >>>>>>>>>> custom
> >>>>>>>>>>>>>> thing,
> >>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>> we changed to integrate with existing
> >>>>>> conventions.
> >>>>>>>>>>>>>> Unfortunately,
> >>>>>>>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>> through the code, I think that the actual
> >>>>>>>> functional
> >>>>>>>>>>>> plumbing
> >>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>> lost
> >>>>>>>>>>>>>>>>>>>>>>>> in the course of multiple refactors
> >>>> around the
> >>>>>>>> call
> >>>>>>>>>>>> context
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> resolver. I
> >>>>>>>>>>>>>>>>>>>>>>>> don't see references to that property or
> >>>> the
> >>>>>>>>>> underlying
> >>>>>>>>>>>>>> header.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id
> >>>> header
> >>>>>>>> feels
> >>>>>>>>>> like
> >>>>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> should definitely keep, especially when
> >>>>>> Polaris is
> >>>>>>>>> one
> >>>>>>>>>>>>>> service in a
> >>>>>>>>>>>>>>>>>>>>>> mesh
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> services that maybe don't have OTel
> >>>>>> integration.
> >>>>>>>>> I'm a
> >>>>>>>>>>>> fan of
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>>>> standard, but it's not entirely
> >>>> ubiquitous and
> >>>>>>>> there
> >>>>>>>>>> are
> >>>>>>>>>>>> many
> >>>>>>>>>>>>>>>>>>>>>> middleware
> >>>>>>>>>>>>>>>>>>>>>>>> layers that know how to forward on the
> >>>>>>>> x-request-id
> >>>>>>>>>>>> header.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
> >>>> Stupp <
> >>>>>>>>>>>> snazy@snazy.de>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability
> >>>> with
> >>>>>> the
> >>>>>>>>>> existing
> >>>>>>>>>>>>>> de-facto
> >>>>>>>>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users
> >>>> to
> >>>>>>>>> integrate
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>> observability platforms.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> >>>>>>>>> Bourlatchkov <
> >>>>>>>>>>>>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Alex and All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I certainly support the idea of
> >>>> following
> >>>>>> OTel
> >>>>>>>>>>>> standards for
> >>>>>>>>>>>>>>>>>>>>>>> achieving
> >>>>>>>>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests
> >>>> and/or
> >>>>>>>> events.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> As to what form the correlation data
> >>>> should
> >>>>>>>> take,
> >>>>>>>>> I
> >>>>>>>>>>>> believe
> >>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context"
> >>>>>> represents.
> >>>>>>>>>> So, I
> >>>>>>>>>>>>>> believe it
> >>>>>>>>>>>>>>>>>>>>>>> makes
> >>>>>>>>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard
> >>>>>> context
> >>>>>>>>>>>> propagators
> >>>>>>>>>>>>>> at the
> >>>>>>>>>>>>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>>>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
> >>>>>>>>>> information,
> >>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> returning
> >>>>>>>>>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>>>>>>> other "correlation" data in the
> >>>> response is
> >>>>>>>>>> redundant, I
> >>>>>>>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request does not have
> >>>> OTel
> >>>>>>>> context
> >>>>>>>>>> info,
> >>>>>>>>>>>>>> what is
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
> >>>>>>>>>> "correlation
> >>>>>>>>>>>> ID"?
> >>>>>>>>>>>>>> How is
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> envisioned to be used?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If the intention is to correlate a
> >>>> Polaris
> >>>>>>>>> response
> >>>>>>>>>>>>>> (operation)
> >>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>> events
> >>>>>>>>>>>>>>>>>>>>>>>>>> that might have resulted from its
> >>>>>> execution, I
> >>>>>>>>>> believe a
> >>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>> robust
> >>>>>>>>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel
> >>>>>> trace
> >>>>>>>> info
> >>>>>>>>>>>>>> (starting a new
> >>>>>>>>>>>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>> necessary) into event data. Then,
> >>>> Polaris
> >>>>>> can
> >>>>>>>> also
> >>>>>>>>>>>> return
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>>>>> context
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a
> >>>> bit
> >>>>>>>> awkward
> >>>>>>>>>> from
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>>>>>> perspective, but might be an option for
> >>>>>>>> supporting
> >>>>>>>>>>>> custom
> >>>>>>>>>>>>>>>>>>>>>>> correlators.
> >>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The
> >>>>>> header
> >>>>>>>>> name
> >>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace
> >>>> Context.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to
> >>>>>> extract
> >>>>>>>>> the
> >>>>>>>>>>>> trace
> >>>>>>>>>>>>>> ID from
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> response and from events and find
> >>>> related
> >>>>>> data.
> >>>>>>>>>>>> Granted, it
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>> require
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to
> >>>>>> decode
> >>>>>>>>> trace
> >>>>>>>>>> IDs
> >>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>>>>>> context, but the format is standard and
> >>>> not
> >>>>>>>>>> complex. The
> >>>>>>>>>>>>>> benefit
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily
> >>>>>> integrate
> >>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>> OTel-compatible
> >>>>>>>>>>>>>>>>>>>>>>>>>> observability platforms regardless of
> >>>>>> whether
> >>>>>>>> any
> >>>>>>>>>>>> particular
> >>>>>>>>>>>>>>>>>>>>>>> deployment
> >>>>>>>>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
> >>>> Alexandre
> >>>>>> Dutra
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>> adutra@apache.org
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of
> >>>> "request
> >>>>>> ID",
> >>>>>>>>> but
> >>>>>>>>>> its
> >>>>>>>>>>>>>> purpose is
> >>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
> >>>>>>>>>> observability
> >>>>>>>>>>>>>> feature to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR
> >>>> aims
> >>>>>> to
> >>>>>>>>>> rename it
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> "correlation
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry
> >>>>>>>> standards
> >>>>>>>>>> [1].
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light
> >>>>>> overlaps
> >>>>>>>>> with
> >>>>>>>>>>>> core
> >>>>>>>>>>>>>>>>>>>>>> telemetry
> >>>>>>>>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is
> >>>>>> enabled
> >>>>>>>> in
> >>>>>>>>>>>> Polaris,
> >>>>>>>>>>>>>> each
> >>>>>>>>>>>>>>>>>>>>>>>>>>> request already has a trace ID and
> >>>> span ID,
> >>>>>>>>> making
> >>>>>>>>>> a
> >>>>>>>>>>>>>> separate
> >>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and
> >>>> span
> >>>>>> ID
> >>>>>>>> in
> >>>>>>>>>>>> Polaris
> >>>>>>>>>>>>>> events,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the generated correlation
> >>>> ID,
> >>>>>> would
> >>>>>>>>>>>>>> significantly
> >>>>>>>>>>>>>>>>>>>>>>>> simplify
> >>>>>>>>>>>>>>>>>>>>>>>>>>> correlation of events with other
> >>>> traces.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following
> >>>> changes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace
> >>>> ID and
> >>>>>>>> span
> >>>>>>>>>> ID as
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ID for the request, instead of
> >>>> generating a
> >>>>>>>>> random
> >>>>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>>>> ID.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> >>>>>>>> correlation
> >>>>>>>>>> ID
> >>>>>>>>>>>>>> header is
> >>>>>>>>>>>>>>>>>>>>>>>> present
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in the request, use it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions
> >>>> is
> >>>>>> met,
> >>>>>>>>>> generate
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> random
> >>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best
> >>>>>> approach
> >>>>>>>>> when a
> >>>>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>>>> ID
> >>>>>>>>>>>>>>>>>>>>>>>>>>> header is present in the request.
> >>>> However,
> >>>>>> I
> >>>>>>>>>> believe it
> >>>>>>>>>>>>>> would be
> >>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is
> >>>>>> enabled, as
> >>>>>>>>>> OTel
> >>>>>>>>>>>>>> offers a
> >>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server
> >>>> trace
> >>>>>>>>>> propagation,
> >>>>>>>>>>>>>> e.g. W3C
> >>>>>>>>>>>>>>>>>>>>>>>> Trace
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Context propagation [2].
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> [1]:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/pull/2757%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1Oe-25vtt4gLVSMFMtSVNg%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2ohYKOQ8XU5-GBlDo6zYGC&source=gmail-imap&ust=1762469626000000&usg=AOvVaw38kU5KZM7IlDD0Y7UPVYn5
> >>>>>>>>>>>>>>>>>>>>>>>>>>> [2]:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.w3.org/TR/trace-context%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1TRdkzABc_7U-_KZ1MZ59v%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw0cUBLtXCkRoH5pL57naUhY&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3UFfRQLLzFZTMF1vmrxROq
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> >>
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi Dmitri,
Thanks for this - I am understanding what you are saying directly, but I’m not sure I am understanding the implications of those statements. Can you please help clarify the following?
Re: separation of “observability/debuggability” vs “functional” use cases:
I agree that the request ID is not a hard requirement for the correctness of the server or client, in general. Maybe there is a use case that exists, but I am not aware of that either. So what I’m gathering from this is that there is not really a “functional” use case here. If you’re referring to the ability of a upstream system (such as any load balancer) to mark a request ID and have Polaris respect that as “functional” - I’m not sure it’s really all that different than an “observability” use case either. I’m not convinced either way - but if you can expand further on this, it may help me see your point.
Re: propagating request IDs upwards
I don’t find this awkward, TBH. This is how most web systems I’ve ever worked with work. The ability to request a specific Request ID tend to be optional, but receiving back a Request ID (regardless of whether it was generated for you, or you had requested it and it was respected) through a header is quite standard. I don’t see how Polaris is any different - happy to hear why this would be the case.
But even in the case that a client is trying to relate requests for a certain “unit of work” on their end, they have the ability to do so by using the `X-Request-Id` request header and Polaris should respect that value and resend the same request ID back as a response header. This actually makes things even clearer for the client to confirm that their Request ID was indeed respected.
Thoughts?
Best,
Adnan Hemani
> On Oct 30, 2025, at 3:53 PM, Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Adnan,
>
> Re: request ID headers in responses (point 4 in your email):
>
> I believe we need to distinguish observability / debuggability use cases
> from "functional" use of request IDs. By functional I mean clients that
> actually rely on the request ID value in order to perform their job
> properly.
>
> I do believe we're discussing the first use case in this thread. Meaning
> that request IDs are auxiliary pieces of information that are not critical
> for correctness of operation in the larger context. However, please correct
> me if I'm wrong here.
>
> That said, for observability / debugging use cases one normally establishes
> the request ID as early as possible and then propagates it to lower-level
> systems. Propagating request IDs upward is awkward because a client making
> multiple calls to Polaris as part of the same logical request on its side
> will end up with different polaris request IDs, so linking them to the
> client-side "unit of work" will require extra machinery.
>
> WDYT?
>
> Thanks,
> Dmitri.
>
> On Thu, Oct 30, 2025 at 5:44 PM Adnan Hemani
> <ad...@snowflake.com.invalid> wrote:
>
>> Hi folks,
>>
>> Unfortunately, I don’t agree on most of the discussion past Alex’s
>> re-summarization. I do, however, agree on the requirements stated by Alex
>> in that email.
>>
>> Here’s what I believe we should be doing in summary:
>> 1. Keep the existing `request_id` column.
>> * To be fair, I don’t fully understand the reason why we are
>> trying to change a ubiquitous concept such as `request_id` to something
>> named as `correlation_id`. Are we trying to state explicitly that the
>> correlation ID and request ID should/are not be the same? If someone can
>> please explain this part, that would be helpful.
>> 2. In the `request_id` field, use the `X-Request-Id` header value if it
>> exists - as that is explicitly what the client is asking for and will mark
>> as the request ID in their logs.
>> 2a. Otherwise, use the OTel Trace ID as the request ID. This
>> should always exist, as it is provided by Quarkus on a per-request basis.
>> 2b. If, in the future, we are not provided an OTel Trace ID (for
>> whatever reason), one should be generated.
>> 4. Keep the `request_id` as part of the response headers.
>> * This allows clients to note down what the request ID that was
>> used and/or generated. Without it, how will clients who don’t provide a
>> `X-Request-Id` header be able to communicate which request they are
>> referring to if they need to debug something?
>> 5. I’m unsure on indexing over request_id or keeping the request_id_type.
>> I don’t see these as a hard use case today, but we can always add it as an
>> improvement in the future.
>>
>> To put this all another way (perhaps more philosophically, but the way I
>> am thinking about things in general):
>> * There should be exactly 1 request ID that is used when a client makes a
>> request to Polaris. That request ID can be dictated by the client (which I
>> still find a bit odd when looking at web server architecture in general,
>> but I understand the use case we have for it) or generated by the server.
>> * That request ID should then be used in any persisting anything that
>> involve request identification (in this particular case, for Events, but
>> not limited to if there’s something else in the future.)
>> * That request ID should be made aware of to the client, so it knows how
>> to identify the request in the case of future debugging.
>>
>> Please let me know your thoughts.
>>
>> Best,
>> Adnan Hemani
>>
>>
>>> On Oct 30, 2025, at 10:26 AM, Dmitri Bourlatchkov <di...@apache.org>
>> wrote:
>>>
>>> Hi Alex,
>>>
>>> SGTM (both not using request ID _response_ headers and deferring type
>> info
>>> in correlation ID values).
>>>
>>> Cheers,
>>> Dmitri.
>>>
>>> On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org>
>> wrote:
>>>
>>>> Hi Dmitri,
>>>>
>>>>> I believe we still need to clarify / confirm whether returning any
>>>> request / correlation ID in _response_ headers is required.
>>>>
>>>> Indeed, that specific item was still pending. FYI I'm going to
>>>> *remove* the echoing of the request ID in the response headers, unless
>>>> someone objects.
>>>>
>>>>> it could be useful to embed some type data into the value to help with
>>>> interpretation
>>>>
>>>> I'm not sure we need this at this moment. I'd suggest that we leave
>>>> this for further improvements in order to not over-specify the
>>>> requirements. Given that the events feature is beta, this shouldn't be
>>>> a problem.
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>>
>>>> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
>>>> wrote:
>>>>>
>>>>> Thanks for the nice summary, Alex!
>>>>>
>>>>> To All:
>>>>>
>>>>> I'm not sure an INDEX on correlation_id is that useful... Polaris
>> itself
>>>>> does not use that index, right? Do we expect end users to query events
>>>>> directly from the database? This will work only with RDBMS, but
>> probably
>>>>> not with the proposed NoSQL persistence.
>>>>>
>>>>> I believe we still need to clarify / confirm whether returning any
>>>> request
>>>>> / correlation ID in _response_ headers is required.
>>>>>
>>>>> From my POV, I do not see a use case for it, but I wonder what other
>>>> people
>>>>> think / need.
>>>>>
>>>>> As for persisting correlation ID type, I agree that we do not need a
>>>>> separate column for that. Still, it could be useful to embed some type
>>>> data
>>>>> into the value to help with interpretation, specifically for telemetry
>>>> use
>>>>> cases (both x-request-id and OTel). For example, we could
>>>>> persist "traceparent:00-12345-...". Then, the reader would be able to
>>>>> identify this as information following the W3C spec [1], parse it and
>>>> link
>>>>> to other traces (if desired). That processing is optional and at the
>>>>> discretion of the event data reader, but making the value no-so-opaque
>>>>> upfront can enable a lot of interesting options for users without much
>>>>> extra overhead in Polaris. WDYT?
>>>>>
>>>>> [1]
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/2021/REC-trace-context-1-20211123/%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw1K633xlm1Dj0NjcNcpd41A&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3FJYjDvXW1ifT-dq78bLOy
>>>>>
>>>>> Thanks,
>>>>> Dmitri.
>>>>>
>>>>> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>>> we would only persist one or the other, not both.
>>>>>>
>>>>>> My apologies, I confess that this wasn't my understanding of the
>>>> desired
>>>>>> design.
>>>>>>
>>>>>> Let's try to gather the requirements again:
>>>>>>
>>>>>> 1. Only one correlation ID is enough
>>>>>> 2. The correlation ID is an opaque string
>>>>>> 3. The main use case is to find events with matching correlation IDs
>>>>>> 4. The only query pattern is by exact match
>>>>>>
>>>>>> Regarding *where* to store the correlation ID, and expanding on what
>>>>>> Dmitri said: the requirement to find matching records by correlation
>>>>>> ID clearly suggests a separate column, and with an INDEX on it.
>>>>>> Storing it in additional_properties will be inefficient.
>>>>>>
>>>>>> If we all agree on the above, here is what I propose:
>>>>>>
>>>>>> 1. Rename the existing column request_id to correlation_id;
>>>>>> 2. Populate from OTel context if available; the exact syntax is
>>>>>> implementation-specific;
>>>>>> 3. Otherwise, populate from X-Request-Id header if available;
>>>>>> 4. Otherwise, don't populate.
>>>>>> 5. Create an INDEX on correlation_id.
>>>>>>
>>>>>> You'll notice that we don't store the ID type, but that doesn't seem
>>>>>> necessary.
>>>>>>
>>>>>> What do you all think?
>>>>>>
>>>>>> Thanks,
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <dimas@apache.org
>>>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Yufei,
>>>>>>>
>>>>>>> [...] tying a BeforeUpdateTable event to the corresponding
>>>>>> AfterUpdateTable
>>>>>>> event.
>>>>>>>
>>>>>>>
>>>>>>> It looks like this calls for a strong and always present ID in
>>>>>> events.This
>>>>>>> ID is obviously distincts from OTel context information and should be
>>>>>>> persisted separately, IMHO.
>>>>>>>
>>>>>>> I believe there's still an open question whether the request ID in
>>>> the
>>>>>>> events should be populated from the x-request-id in REST API headers.
>>>>>>>
>>>>>>> Yufei: Is that required in your use cases?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dmitri.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>>> I think the first question is whether any downstream use cases
>>>> need to
>>>>>>>> interpret the semantics of a request ID or OTel context. As I
>>>>>> mentioned,
>>>>>>>> the original purpose of a request ID was event correlation, e.g.,
>>>>>> tying a
>>>>>>>> BeforeUpdateTable event to the corresponding AfterUpdateTable
>>>> event. If
>>>>>>>> that’s the only requirement, an opaque correlation ID is good
>>>> enough.
>>>>>> We
>>>>>>>> could change the field name(request-id) if needed.
>>>>>>>>
>>>>>>>> If we do have use cases that depend on the meaning of the request
>>>> ID or
>>>>>>>> OTel context, we can store them in additional_properties (
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%2523L134-L134%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3kHvvqJhJTJi61gR6c6CFy&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0kUL91EJEo0g8DuALtlFaF
>>>>>>>> ).
>>>>>>>> This is more flexible because it allows us to:
>>>>>>>> 1. Record a key that indicates the format (e.g., x-request-id vs.
>>>>>>>> otel-context).
>>>>>>>> 2. Preserve both when Polaris receives both headers.
>>>>>>>> 3. Add future tracing fields without a schema change.
>>>>>>>>
>>>>>>>> Yufei
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
>>>> dimas@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> From my POV the use cases for x-request-id and OTel are slightly
>>>>>>>> different.
>>>>>>>>> OTel is an observability use case where cross-linking of various
>>>>>> pieces
>>>>>>>> of
>>>>>>>>> data is desired but not required in all cases (data loss is
>>>>>> acceptable).
>>>>>>>>> The x-request-id use case appears to have some very specific
>>>> custom
>>>>>>>>> applications without any obvious relationship to OTel.
>>>>>>>>>
>>>>>>>>> Given the current state of this discussion, I think it makes
>>>> sense to
>>>>>>>>> handle and persist them separately.
>>>>>>>>>
>>>>>>>>> However, if we intend x-request-id to act as an observability aid
>>>>>>>>> alternative to OTel, then it might be preferable to have a config
>>>>>> flag
>>>>>>>>> indicating which one of them should be added to persistent event
>>>>>> data. In
>>>>>>>>> that case we should reuse the same column for them (and possibly
>>>> add
>>>>>> a
>>>>>>>>> "type" prefix, e.g. "x-request-id:ABCDEF" or
>>>>>> "traceparent:00-12345...")
>>>>>>>>>
>>>>>>>>> WDYT?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Dmitri.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
>>>> adutra@apache.org>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Yufei,
>>>>>>>>>>
>>>>>>>>>>> I’d suggest persisting a single opaque correlation ID instead
>>>>>>>>>>
>>>>>>>>>> The problem I see with this is that readers cannot know the
>>>> syntax
>>>>>> of
>>>>>>>>>> the opaque id, and will have to "probe" using some heuristics,
>>>>>> e.g. if
>>>>>>>>>> the id matches the OTel trace context syntax, asume it's OTel,
>>>>>>>>>> otherwise... assume it's something else.
>>>>>>>>>>
>>>>>>>>>> That's not unfeasible, but I wonder if it isn't better to store
>>>>>> the 2
>>>>>>>>>> ids separately.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
>>>>>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Thanks Alex for raising this discussion. Thanks everyone for
>>>>>> chiming
>>>>>>>>> in.
>>>>>>>>>>> The plan looks good to me overall.
>>>>>>>>>>>
>>>>>>>>>>>> Update PolarisEvent, expose request ID if available, expose
>>>>>> OTel
>>>>>>>>>> context
>>>>>>>>>>> if available.
>>>>>>>>>>>
>>>>>>>>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2914%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw33p8lOR4oYEJqcbo4Sm74f&source=gmail-imap&ust=1762469626000000&usg=AOvVaw1aWh6q_9-TX1Ao82MTOmiZ
>> persists both
>>>>>> request id
>>>>>>>>> and
>>>>>>>>>>> otel context.
>>>>>>>>>>>
>>>>>>>>>>> Do we actually need to persist both?
>>>>>>>>>>> From the original discussion, the intent of a request ID was
>>>> to
>>>>>>>>> correlate
>>>>>>>>>>> multiple events generated by the same request, for example,
>>>>>> linking a
>>>>>>>>>>> BeforeUpdateTable and an AfterUpdateTable event. In that
>>>> sense,
>>>>>> what
>>>>>>>> we
>>>>>>>>>>> really need is a single opaque identifier that ties related
>>>>>> events
>>>>>>>>>>> together, regardless of whether it comes from a request ID
>>>> or an
>>>>>> OTel
>>>>>>>>>>> context.
>>>>>>>>>>>
>>>>>>>>>>> If we persist both fields (requestId and otelContext), it can
>>>>>> lead to
>>>>>>>>>>> consumer confusion, since downstream systems would need to
>>>> decide
>>>>>>>> which
>>>>>>>>>> one
>>>>>>>>>>> to rely on, potentially causing inconsistent event
>>>> correlation.
>>>>>>>> Unless
>>>>>>>>>>> there are concrete use cases where event consumers need
>>>> both, I’d
>>>>>>>>> suggest
>>>>>>>>>>> persisting a single opaque correlation ID instead. This
>>>> keeps the
>>>>>>>>> schema
>>>>>>>>>>> simpler and ensures consistent event linking.
>>>>>>>>>>>
>>>>>>>>>>> Yufei
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
>>>>>> adutra@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> Here is the PR:
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2914%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw33p8lOR4oYEJqcbo4Sm74f&source=gmail-imap&ust=1762469626000000&usg=AOvVaw1aWh6q_9-TX1Ao82MTOmiZ
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Alex
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
>>>>>>>>> jb@nanthrax.net>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah it looks good to me and I agree we have a consensus.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> JB
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
>>>>>> adutra@apache.org
>>>>>>>>>
>>>>>>>>> a
>>>>>>>>>>>> écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the good discussion so far. I think we have
>>>> a
>>>>>> broad
>>>>>>>>>>>> consensus.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Unless anybody disagrees I am going to move forward
>>>> with
>>>>>> the
>>>>>>>> the
>>>>>>>>>>>>>> following revised plan, incorporating the latest
>>>> feedback:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. Restore the original functionality for request IDs,
>>>>>> change
>>>>>>>> the
>>>>>>>>>>>>>> default header name back to x-request-id
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3. Update PolarisEvent, expose request ID if available,
>>>>>> expose
>>>>>>>>> OTel
>>>>>>>>>>>>>> context if available.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 4. Update events table SQL schema: insert request ID if
>>>>>>>>> available,
>>>>>>>>>>>>>> insert OTel context if available.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
>>>>>>>>>> jb@nanthrax.net
>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think it's better to use w3c_trace_context (better
>>>>>> than two
>>>>>>>>>>>> fields).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regarding x-request-id, I think it's for a different
>>>>>> purpose,
>>>>>>>>>>>>>>> specifically for persistence. Personally, I think we
>>>> can
>>>>>>>>> achieve
>>>>>>>>>> the
>>>>>>>>>>>>>>> same with span and events.
>>>>>>>>>>>>>>> If we want to "distinguish" the two layers, it makes
>>>>>> sense to
>>>>>>>>>> have
>>>>>>>>>>>>>>> x-request-id (not sure it's super helpful). Else, I
>>>>>> think we
>>>>>>>>> can
>>>>>>>>>>>>>>> achieve the same with span/event.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just my $0.01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested
>>>>>> changes as
>>>>>>>>>> well.
>>>>>>>>>>>> Great
>>>>>>>>>>>>>> find, Dmitri!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I’m still debating with myself on whether we need
>>>> to
>>>>>> store
>>>>>>>>> the
>>>>>>>>>>>>>> `x-request-id` field as part of the Events persistence.
>>>>>> Can we
>>>>>>>>>> think
>>>>>>>>>>>> of a
>>>>>>>>>>>>>> good use case where this is more helpful to the user
>>>> than
>>>>>> the
>>>>>>>>> OTel
>>>>>>>>>>>>>> Trace/Span IDs? I am making the assumption here that
>>>> those
>>>>>> are
>>>>>>>>>> still
>>>>>>>>>>>> being
>>>>>>>>>>>>>> returned back to the client.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
>>>>>>>>>> adutra@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Dmitri,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yes, I think your suggestion to use just one
>>>> field
>>>>>>>>>>>> w3c_trace_context
>>>>>>>>>>>>>>>>> makes more sense than two fields (span_id and
>>>>>> trace_id).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> With that, I also think that we are slowly
>>>> drifting
>>>>>> into
>>>>>>>>>>>>>>>>> implementation considerations; let's get
>>>> consensus
>>>>>> on the
>>>>>>>>>> general
>>>>>>>>>>>>>>>>> design first, and we can certainly fine-tune the
>>>>>> actual
>>>>>>>>> Java
>>>>>>>>>>>> methods
>>>>>>>>>>>>>>>>> and SQL table columns in the future PR. WDYT?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri
>>>> Bourlatchkov <
>>>>>>>>>>>>>> dimas@apache.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Many thanks for the background info, Adnan!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 to action items proposed by Alex!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Re: (3) Can we abstract request/trace info into
>>>> a
>>>>>>>> separate
>>>>>>>>>>>> object
>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>> exposing those accessors on the Event class
>>>>>> directly?
>>>>>>>> OTel
>>>>>>>>>>>> defines
>>>>>>>>>>>>>>>>>> trace/span concepts, but in request ID is a bit
>>>>>> foreign
>>>>>>>> to
>>>>>>>>>> OTel.
>>>>>>>>>>>>>> Having
>>>>>>>>>>>>>>>>>> tracing / request ID isolated in java could help
>>>>>> with
>>>>>>>>>>>> maintaining
>>>>>>>>>>>>>> it and
>>>>>>>>>>>>>>>>>> potentially supporting other (custom) tracing
>>>>>> methods.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Re: (4) I'd like to propose storing OTel
>>>> correlation
>>>>>>>> data
>>>>>>>>>> in the
>>>>>>>>>>>>>> form of a
>>>>>>>>>>>>>>>>>> standard context propagation string, e.g. W3C
>>>>>>>>> trace-context
>>>>>>>>>> [1]
>>>>>>>>>>>>>> (same value
>>>>>>>>>>>>>>>>>> as its HTTP header), so the column could be
>>>> called
>>>>>>>>>>>>>> w3c_trace_context or
>>>>>>>>>>>>>>>>>> simply trace_context.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Open question: do we need to write a separate,
>>>>>>>> individual
>>>>>>>>>> trace
>>>>>>>>>>>> ID
>>>>>>>>>>>>>> field in
>>>>>>>>>>>>>>>>>> SQL? I suppose it is not very useful since
>>>>>> correlating
>>>>>>>> it
>>>>>>>>> to
>>>>>>>>>>>> other
>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>> data already requires understanding OTel context
>>>>>>>>> propagation
>>>>>>>>>>>> and a
>>>>>>>>>>>>>> query
>>>>>>>>>>>>>>>>>> against trace_context can still be made using
>>>>>>>>>> string-matching
>>>>>>>>>>>>>> clauses. We
>>>>>>>>>>>>>>>>>> could probably (additionally) store it in the
>>>>>> request_id
>>>>>>>>>> column
>>>>>>>>>>>> if
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> Polaris-specific request ID header is not set.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As for span ID, I do not really see a use case
>>>> for
>>>>>>>>>> persisting it
>>>>>>>>>>>>>>>>>> individually. It is very specific to OTel trace
>>>> data
>>>>>>>>>>>> construction.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Actually, using the W3C trace context [1]
>>>> encoding
>>>>>>>>> probably
>>>>>>>>>>>> makes
>>>>>>>>>>>>>> sense in
>>>>>>>>>>>>>>>>>> the java event representation too. Interested
>>>>>> callers
>>>>>>>> can
>>>>>>>>>> easily
>>>>>>>>>>>>>> decode
>>>>>>>>>>>>>>>>>> this information since the format is
>>>> well-defined.
>>>>>> As a
>>>>>>>>> side
>>>>>>>>>>>>>> benefit, this
>>>>>>>>>>>>>>>>>> opens opportunities for downstream event
>>>> consumers
>>>>>> to
>>>>>>>>>> connect
>>>>>>>>>>>>>> (propagate
>>>>>>>>>>>>>>>>>> context) to traces that produced events based
>>>> on the
>>>>>>>> event
>>>>>>>>>> data
>>>>>>>>>>>>>> itself,
>>>>>>>>>>>>>>>>>> without relying on the intermediate frameworks.
>>>>>> This may
>>>>>>>>> be
>>>>>>>>>>>>>> desirable since
>>>>>>>>>>>>>>>>>> the current Polaris event persistence impl.
>>>> writes
>>>>>>>> events
>>>>>>>>> in
>>>>>>>>>>>>>> batches, so
>>>>>>>>>>>>>>>>>> the association to individual requests that
>>>> produced
>>>>>>>> those
>>>>>>>>>>>> events
>>>>>>>>>>>>>> is lost.
>>>>>>>>>>>>>>>>>> Whether to perform this kind of context
>>>> propagation
>>>>>> will
>>>>>>>>> be
>>>>>>>>>> at
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> discretion of the event consumer, of course
>>>>>> (outside of
>>>>>>>>>> Polaris
>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context/%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw2fn0lRsTx-f9r4PCz8wmJK%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw0JHBUVrCE7tookVBiFjNdP&source=gmail-imap&ust=1762469626000000&usg=AOvVaw2tTZYzZmafSmGBVhT4wv7Z
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
>>>> <
>>>>>>>>>>>> adutra@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you for chiming in; the context around
>>>>>> request
>>>>>>>> IDs
>>>>>>>>>> is now
>>>>>>>>>>>>>> clear.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Trying to summarize this thread into actionable
>>>>>> items,
>>>>>>>>>> here's
>>>>>>>>>>>> what
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> propose:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. Restore the original functionality for
>>>> request
>>>>>> IDs.
>>>>>>>>>>>>>>>>>>> - Change the default header name back to
>>>>>>>> x-request-id
>>>>>>>>>>>> (despite
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> x- prefix being deprecated), but keep it
>>>>>> configurable
>>>>>>>> as
>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related
>>>>>> functionality.
>>>>>>>>>>>>>>>>>>> - Do not generate a request ID if the REST
>>>>>> client
>>>>>>>>>> doesn't
>>>>>>>>>>>>>> provide one.
>>>>>>>>>>>>>>>>>>> 3. Update PolarisEvent:
>>>>>>>>>>>>>>>>>>> - Expose new requestId(), traceId(),
>>>> spanId()
>>>>>>>> methods,
>>>>>>>>>> all
>>>>>>>>>>>>>> nullable.
>>>>>>>>>>>>>>>>>>> - This would align with the emerging
>>>> consensus
>>>>>>>> around
>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>> contextual information in PolarisEvent [1].
>>>>>>>>>>>>>>>>>>> 4. Update events table SQL schema:
>>>>>>>>>>>>>>>>>>> - Insert the client-provided request ID
>>>> into the
>>>>>>>>>> request_id
>>>>>>>>>>>>>>>>>>> column; otherwise, insert null.
>>>>>>>>>>>>>>>>>>> - Add two new nullable columns, trace_id and
>>>>>>>> span_id,
>>>>>>>>>> and
>>>>>>>>>>>>>> populate
>>>>>>>>>>>>>>>>>>> them if OTel is enabled.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> From our discussions, I think it's important
>>>> to not
>>>>>>>>>> conflate
>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>> tracing fields with Envoy tracing fields,
>>>> which is
>>>>>> why
>>>>>>>> I
>>>>>>>>>>>> suggest we
>>>>>>>>>>>>>>>>>>> use separate fields / columns for them.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Would the above plan work for everyone?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw02wPvb0qxRzYAKEP0h8l9T%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3AIjY12CwTp7ItIvzTOIIN&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0O6N5bjiJtEUkTDiZL-DlH
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
>>>>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks to Alex for starting this thread -
>>>> because
>>>>>> of
>>>>>>>>>> this, I’m
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>> coming to the realization that OTel Trace and
>>>> Span
>>>>>> IDs
>>>>>>>>> are
>>>>>>>>>>>> coming
>>>>>>>>>>>>>> built-in
>>>>>>>>>>>>>>>>>>> with Quarkus and my previous work to generate a
>>>>>>>>>>>>>> Request/Correlation ID is
>>>>>>>>>>>>>>>>>>> likely not needed as a result. My original
>>>>>> motivation
>>>>>>>> for
>>>>>>>>>>>>>> generation of a
>>>>>>>>>>>>>>>>>>> Request/Correlation ID was to ensure that any
>>>>>> client
>>>>>>>> can
>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>> identify
>>>>>>>>>>>>>>>>>>> a request made to Polaris, which would be
>>>>>> especially
>>>>>>>>>> useful for
>>>>>>>>>>>>>> debugging
>>>>>>>>>>>>>>>>>>> failing requests or identifying call patterns.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we
>>>>>> should
>>>>>>>>>> remove
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Request/Correlation ID generation and always
>>>> use
>>>>>> the
>>>>>>>> OTel
>>>>>>>>>>>>>> trace/span IDs
>>>>>>>>>>>>>>>>>>> (which come for free with Quarkus) instead for
>>>> the
>>>>>>>>>> Correlation
>>>>>>>>>>>> ID
>>>>>>>>>>>>>> unless a
>>>>>>>>>>>>>>>>>>> valid header is present, which would take over
>>>> as
>>>>>> the
>>>>>>>>>>>> Correlation
>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>> instead.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> To answer Dmitri’s question re Polaris
>>>> Events: The
>>>>>>>>>> intended
>>>>>>>>>>>> use
>>>>>>>>>>>>>> case is
>>>>>>>>>>>>>>>>>>> to provide some sort of correlation between
>>>> events
>>>>>> that
>>>>>>>>>> have
>>>>>>>>>>>>>> occurred as
>>>>>>>>>>>>>>>>>>> part of the same request. For example, if a
>>>> user
>>>>>> makes
>>>>>>>> an
>>>>>>>>>>>>>> CommitTransaction
>>>>>>>>>>>>>>>>>>> request, it would be helpful to see all
>>>> UpdateTable
>>>>>>>> calls
>>>>>>>>>> that
>>>>>>>>>>>>>> were made as
>>>>>>>>>>>>>>>>>>> part of that one user request.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
>>>>>> Bourlatchkov <
>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Just to confirm: if / when we restore that,
>>>>>> Polaris
>>>>>>>>> will
>>>>>>>>>> NOT
>>>>>>>>>>>>>> generate
>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>> IDs in case the header is not present in the
>>>>>> request,
>>>>>>>>>>>> correct?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I believe x-request-id can co-exist with
>>>> OTel.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What about adding request IDs to events
>>>> [1][2]?
>>>>>>>> What's
>>>>>>>>>> the
>>>>>>>>>>>>>> intended use
>>>>>>>>>>>>>>>>>>>>> case for that? Could you share some context
>>>> here
>>>>>> too?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event
>>>>>>>> persistence
>>>>>>>>>> as
>>>>>>>>>>>>>> "beta" in
>>>>>>>>>>>>>>>>>>>>> 1.2.0... This discussion adds another point
>>>>>> towards
>>>>>>>>>> that, I
>>>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%25252523L97%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1Pioys4ROm8vYM_mdx4ygH%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw39NasMg4Tgyrfe5Bj_U4xY&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0sDeqLAsJrlT0vl5OSGLZC
>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%25252523L66%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw12-7e3ahm2sLSkLSNqTecm%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1PGW6uFwtUN8F1dOlJwrpm%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw3y_SObfpEJH_K_8RD9lVTL&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0Iwahb3E3xhtLPtdPmqs1i
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> [2877]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/pull/2877%25252523discussion_r2456300613%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2_e3B3qU8kJxdMmQx-q6bZ&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3BMSIO2LpE1BaeIgK-c9nY
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
>>>> Collado <
>>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hey Dmitri
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The generating a request id is new code
>>>> that was
>>>>>>>> added
>>>>>>>>>>>> after the
>>>>>>>>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>>>> x-request-id support. You can see the state
>>>>>> from ~1
>>>>>>>>> year
>>>>>>>>>>>> ago, we
>>>>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>>>>>>> request_id as the header we used for the
>>>> MDC -
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%25252523L415-L416%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw34DksOw7DSHJu8PfQQ5bT6%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw1g09FwhrTquJs9utV9CAX9&source=gmail-imap&ust=1762469626000000&usg=AOvVaw0dwBsX019LeuexMDdck0dM
>>>>>>>>>>>>>>>>>>>>>> . At some point, it was changed to be
>>>>>> configurable,
>>>>>>>>>> then the
>>>>>>>>>>>>>>>>>>>>>> ContextResolverFilter filter was
>>>>>>>> refactored/eliminated
>>>>>>>>>> and
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but
>>>>>> lost
>>>>>>>> some
>>>>>>>>>> of
>>>>>>>>>>>> its
>>>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>>>> functionality.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> My personal opinion is that restoring
>>>> support
>>>>>> for
>>>>>>>> the
>>>>>>>>>>>>>> x-request-id
>>>>>>>>>>>>>>>>>>> header
>>>>>>>>>>>>>>>>>>>>>> is something that we should do, but if the
>>>>>> header
>>>>>>>>> isn't
>>>>>>>>>>>> present,
>>>>>>>>>>>>>>>>>>> falling
>>>>>>>>>>>>>>>>>>>>>> back on simply using OTel trace ids is good
>>>>>> enough
>>>>>>>>>> (better,
>>>>>>>>>>>>>> even) than
>>>>>>>>>>>>>>>>>>>>>> generating another random request id.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
>>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks for the info!
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes
>>>>>> sense to
>>>>>>>>> me.
>>>>>>>>>>>>>> However, I
>>>>>>>>>>>>>>>>>>>>>> wonder:
>>>>>>>>>>>>>>>>>>>>>>> why would Polaris need to generate a new
>>>>>> request ID
>>>>>>>>>> inside
>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>> code?..
>>>>>>>>>>>>>>>>>>>>>>> and return it in response headers?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> How important is it to propagate this ID to
>>>>>> Polaris
>>>>>>>>>> Events?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm just trying to understand the full
>>>> context
>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
>>>>>> Collado <
>>>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I think the original intention for this
>>>>>> requestId
>>>>>>>>>> field
>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>> request propagation from load balancers,
>>>> like
>>>>>>>> Envoy
>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw3LNqtilZ2OoJfE7yEwraZa%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2hTMXu2DU30Lg_5UBzsDJw&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3EG5R-XtDVNQoOZTJYC4BU
>>>>>>>>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask
>>>> me
>>>>>> why
>>>>>>>> the
>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was
>>>>>> originally
>>>>>>>> a
>>>>>>>>>> custom
>>>>>>>>>>>>>> thing,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>> we changed to integrate with existing
>>>>>> conventions.
>>>>>>>>>>>>>> Unfortunately,
>>>>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>> through the code, I think that the actual
>>>>>>>> functional
>>>>>>>>>>>> plumbing
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>> lost
>>>>>>>>>>>>>>>>>>>>>>>> in the course of multiple refactors
>>>> around the
>>>>>>>> call
>>>>>>>>>>>> context
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> resolver. I
>>>>>>>>>>>>>>>>>>>>>>>> don't see references to that property or
>>>> the
>>>>>>>>>> underlying
>>>>>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id
>>>> header
>>>>>>>> feels
>>>>>>>>>> like
>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> should definitely keep, especially when
>>>>>> Polaris is
>>>>>>>>> one
>>>>>>>>>>>>>> service in a
>>>>>>>>>>>>>>>>>>>>>> mesh
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> services that maybe don't have OTel
>>>>>> integration.
>>>>>>>>> I'm a
>>>>>>>>>>>> fan of
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>> standard, but it's not entirely
>>>> ubiquitous and
>>>>>>>> there
>>>>>>>>>> are
>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>>>>>> middleware
>>>>>>>>>>>>>>>>>>>>>>>> layers that know how to forward on the
>>>>>>>> x-request-id
>>>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
>>>> Stupp <
>>>>>>>>>>>> snazy@snazy.de>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability
>>>> with
>>>>>> the
>>>>>>>>>> existing
>>>>>>>>>>>>>> de-facto
>>>>>>>>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users
>>>> to
>>>>>>>>> integrate
>>>>>>>>>>>> into
>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>> observability platforms.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
>>>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Alex and All,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I certainly support the idea of
>>>> following
>>>>>> OTel
>>>>>>>>>>>> standards for
>>>>>>>>>>>>>>>>>>>>>>> achieving
>>>>>>>>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests
>>>> and/or
>>>>>>>> events.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> As to what form the correlation data
>>>> should
>>>>>>>> take,
>>>>>>>>> I
>>>>>>>>>>>> believe
>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context"
>>>>>> represents.
>>>>>>>>>> So, I
>>>>>>>>>>>>>> believe it
>>>>>>>>>>>>>>>>>>>>>>> makes
>>>>>>>>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard
>>>>>> context
>>>>>>>>>>>> propagators
>>>>>>>>>>>>>> at the
>>>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
>>>>>>>>>> information,
>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> returning
>>>>>>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>>>> other "correlation" data in the
>>>> response is
>>>>>>>>>> redundant, I
>>>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request does not have
>>>> OTel
>>>>>>>> context
>>>>>>>>>> info,
>>>>>>>>>>>>>> what is
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
>>>>>>>>>> "correlation
>>>>>>>>>>>> ID"?
>>>>>>>>>>>>>> How is
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> envisioned to be used?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If the intention is to correlate a
>>>> Polaris
>>>>>>>>> response
>>>>>>>>>>>>>> (operation)
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>> events
>>>>>>>>>>>>>>>>>>>>>>>>>> that might have resulted from its
>>>>>> execution, I
>>>>>>>>>> believe a
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> robust
>>>>>>>>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel
>>>>>> trace
>>>>>>>> info
>>>>>>>>>>>>>> (starting a new
>>>>>>>>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>> necessary) into event data. Then,
>>>> Polaris
>>>>>> can
>>>>>>>> also
>>>>>>>>>>>> return
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>>>> context
>>>>>>>>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a
>>>> bit
>>>>>>>> awkward
>>>>>>>>>> from
>>>>>>>>>>>> the
>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>>>> perspective, but might be an option for
>>>>>>>> supporting
>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>>>>>>>> correlators.
>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The
>>>>>> header
>>>>>>>>> name
>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace
>>>> Context.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to
>>>>>> extract
>>>>>>>>> the
>>>>>>>>>>>> trace
>>>>>>>>>>>>>> ID from
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> response and from events and find
>>>> related
>>>>>> data.
>>>>>>>>>>>> Granted, it
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to
>>>>>> decode
>>>>>>>>> trace
>>>>>>>>>> IDs
>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>>>> context, but the format is standard and
>>>> not
>>>>>>>>>> complex. The
>>>>>>>>>>>>>> benefit
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily
>>>>>> integrate
>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> OTel-compatible
>>>>>>>>>>>>>>>>>>>>>>>>>> observability platforms regardless of
>>>>>> whether
>>>>>>>> any
>>>>>>>>>>>> particular
>>>>>>>>>>>>>>>>>>>>>>> deployment
>>>>>>>>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
>>>> Alexandre
>>>>>> Dutra
>>>>>>>> <
>>>>>>>>>>>>>>>>>>> adutra@apache.org
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of
>>>> "request
>>>>>> ID",
>>>>>>>>> but
>>>>>>>>>> its
>>>>>>>>>>>>>> purpose is
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
>>>>>>>>>> observability
>>>>>>>>>>>>>> feature to
>>>>>>>>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR
>>>> aims
>>>>>> to
>>>>>>>>>> rename it
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> "correlation
>>>>>>>>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry
>>>>>>>> standards
>>>>>>>>>> [1].
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light
>>>>>> overlaps
>>>>>>>>> with
>>>>>>>>>>>> core
>>>>>>>>>>>>>>>>>>>>>> telemetry
>>>>>>>>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is
>>>>>> enabled
>>>>>>>> in
>>>>>>>>>>>> Polaris,
>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>>>>>>>>> request already has a trace ID and
>>>> span ID,
>>>>>>>>> making
>>>>>>>>>> a
>>>>>>>>>>>>>> separate
>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and
>>>> span
>>>>>> ID
>>>>>>>> in
>>>>>>>>>>>> Polaris
>>>>>>>>>>>>>> events,
>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the generated correlation
>>>> ID,
>>>>>> would
>>>>>>>>>>>>>> significantly
>>>>>>>>>>>>>>>>>>>>>>>> simplify
>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation of events with other
>>>> traces.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following
>>>> changes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace
>>>> ID and
>>>>>>>> span
>>>>>>>>>> ID as
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>>>>>>> ID for the request, instead of
>>>> generating a
>>>>>>>>> random
>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>> ID.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
>>>>>>>> correlation
>>>>>>>>>> ID
>>>>>>>>>>>>>> header is
>>>>>>>>>>>>>>>>>>>>>>>> present
>>>>>>>>>>>>>>>>>>>>>>>>>>> in the request, use it.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions
>>>> is
>>>>>> met,
>>>>>>>>>> generate
>>>>>>>>>>>> a
>>>>>>>>>>>>>> random
>>>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best
>>>>>> approach
>>>>>>>>> when a
>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>>>>>>>>>> header is present in the request.
>>>> However,
>>>>>> I
>>>>>>>>>> believe it
>>>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is
>>>>>> enabled, as
>>>>>>>>>> OTel
>>>>>>>>>>>>>> offers a
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server
>>>> trace
>>>>>>>>>> propagation,
>>>>>>>>>>>>>> e.g. W3C
>>>>>>>>>>>>>>>>>>>>>>>> Trace
>>>>>>>>>>>>>>>>>>>>>>>>>>> Context propagation [2].
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://github.com/apache/polaris/pull/2757%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1Oe-25vtt4gLVSMFMtSVNg%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw2ohYKOQ8XU5-GBlDo6zYGC&source=gmail-imap&ust=1762469626000000&usg=AOvVaw38kU5KZM7IlDD0Y7UPVYn5
>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.google.com/url?q%25253Dhttps://www.w3.org/TR/trace-context%252526source%25253Dgmail-imap%252526ust%25253D1761851831000000%252526usg%25253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%2526source%253Dgmail-imap%2526ust%253D1761927167000000%2526usg%253DAOvVaw1TRdkzABc_7U-_KZ1MZ59v%26source%3Dgmail-imap%26ust%3D1762450034000000%26usg%3DAOvVaw0cUBLtXCkRoH5pL57naUhY&source=gmail-imap&ust=1762469626000000&usg=AOvVaw3UFfRQLLzFZTMF1vmrxROq
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Adnan,
Re: request ID headers in responses (point 4 in your email):
I believe we need to distinguish observability / debuggability use cases
from "functional" use of request IDs. By functional I mean clients that
actually rely on the request ID value in order to perform their job
properly.
I do believe we're discussing the first use case in this thread. Meaning
that request IDs are auxiliary pieces of information that are not critical
for correctness of operation in the larger context. However, please correct
me if I'm wrong here.
That said, for observability / debugging use cases one normally establishes
the request ID as early as possible and then propagates it to lower-level
systems. Propagating request IDs upward is awkward because a client making
multiple calls to Polaris as part of the same logical request on its side
will end up with different polaris request IDs, so linking them to the
client-side "unit of work" will require extra machinery.
WDYT?
Thanks,
Dmitri.
On Thu, Oct 30, 2025 at 5:44 PM Adnan Hemani
<ad...@snowflake.com.invalid> wrote:
> Hi folks,
>
> Unfortunately, I don’t agree on most of the discussion past Alex’s
> re-summarization. I do, however, agree on the requirements stated by Alex
> in that email.
>
> Here’s what I believe we should be doing in summary:
> 1. Keep the existing `request_id` column.
> * To be fair, I don’t fully understand the reason why we are
> trying to change a ubiquitous concept such as `request_id` to something
> named as `correlation_id`. Are we trying to state explicitly that the
> correlation ID and request ID should/are not be the same? If someone can
> please explain this part, that would be helpful.
> 2. In the `request_id` field, use the `X-Request-Id` header value if it
> exists - as that is explicitly what the client is asking for and will mark
> as the request ID in their logs.
> 2a. Otherwise, use the OTel Trace ID as the request ID. This
> should always exist, as it is provided by Quarkus on a per-request basis.
> 2b. If, in the future, we are not provided an OTel Trace ID (for
> whatever reason), one should be generated.
> 4. Keep the `request_id` as part of the response headers.
> * This allows clients to note down what the request ID that was
> used and/or generated. Without it, how will clients who don’t provide a
> `X-Request-Id` header be able to communicate which request they are
> referring to if they need to debug something?
> 5. I’m unsure on indexing over request_id or keeping the request_id_type.
> I don’t see these as a hard use case today, but we can always add it as an
> improvement in the future.
>
> To put this all another way (perhaps more philosophically, but the way I
> am thinking about things in general):
> * There should be exactly 1 request ID that is used when a client makes a
> request to Polaris. That request ID can be dictated by the client (which I
> still find a bit odd when looking at web server architecture in general,
> but I understand the use case we have for it) or generated by the server.
> * That request ID should then be used in any persisting anything that
> involve request identification (in this particular case, for Events, but
> not limited to if there’s something else in the future.)
> * That request ID should be made aware of to the client, so it knows how
> to identify the request in the case of future debugging.
>
> Please let me know your thoughts.
>
> Best,
> Adnan Hemani
>
>
> > On Oct 30, 2025, at 10:26 AM, Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> >
> > Hi Alex,
> >
> > SGTM (both not using request ID _response_ headers and deferring type
> info
> > in correlation ID values).
> >
> > Cheers,
> > Dmitri.
> >
> > On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> >> Hi Dmitri,
> >>
> >>> I believe we still need to clarify / confirm whether returning any
> >> request / correlation ID in _response_ headers is required.
> >>
> >> Indeed, that specific item was still pending. FYI I'm going to
> >> *remove* the echoing of the request ID in the response headers, unless
> >> someone objects.
> >>
> >>> it could be useful to embed some type data into the value to help with
> >> interpretation
> >>
> >> I'm not sure we need this at this moment. I'd suggest that we leave
> >> this for further improvements in order to not over-specify the
> >> requirements. Given that the events feature is beta, this shouldn't be
> >> a problem.
> >>
> >> Thanks,
> >> Alex
> >>
> >>
> >> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
> >> wrote:
> >>>
> >>> Thanks for the nice summary, Alex!
> >>>
> >>> To All:
> >>>
> >>> I'm not sure an INDEX on correlation_id is that useful... Polaris
> itself
> >>> does not use that index, right? Do we expect end users to query events
> >>> directly from the database? This will work only with RDBMS, but
> probably
> >>> not with the proposed NoSQL persistence.
> >>>
> >>> I believe we still need to clarify / confirm whether returning any
> >> request
> >>> / correlation ID in _response_ headers is required.
> >>>
> >>> From my POV, I do not see a use case for it, but I wonder what other
> >> people
> >>> think / need.
> >>>
> >>> As for persisting correlation ID type, I agree that we do not need a
> >>> separate column for that. Still, it could be useful to embed some type
> >> data
> >>> into the value to help with interpretation, specifically for telemetry
> >> use
> >>> cases (both x-request-id and OTel). For example, we could
> >>> persist "traceparent:00-12345-...". Then, the reader would be able to
> >>> identify this as information following the W3C spec [1], parse it and
> >> link
> >>> to other traces (if desired). That processing is optional and at the
> >>> discretion of the event data reader, but making the value no-so-opaque
> >>> upfront can enable a lot of interesting options for users without much
> >>> extra overhead in Polaris. WDYT?
> >>>
> >>> [1]
> https://www.google.com/url?q=https://www.w3.org/TR/2021/REC-trace-context-1-20211123/&source=gmail-imap&ust=1762450034000000&usg=AOvVaw1K633xlm1Dj0NjcNcpd41A
> >>>
> >>> Thanks,
> >>> Dmitri.
> >>>
> >>> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
> >> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>>> we would only persist one or the other, not both.
> >>>>
> >>>> My apologies, I confess that this wasn't my understanding of the
> >> desired
> >>>> design.
> >>>>
> >>>> Let's try to gather the requirements again:
> >>>>
> >>>> 1. Only one correlation ID is enough
> >>>> 2. The correlation ID is an opaque string
> >>>> 3. The main use case is to find events with matching correlation IDs
> >>>> 4. The only query pattern is by exact match
> >>>>
> >>>> Regarding *where* to store the correlation ID, and expanding on what
> >>>> Dmitri said: the requirement to find matching records by correlation
> >>>> ID clearly suggests a separate column, and with an INDEX on it.
> >>>> Storing it in additional_properties will be inefficient.
> >>>>
> >>>> If we all agree on the above, here is what I propose:
> >>>>
> >>>> 1. Rename the existing column request_id to correlation_id;
> >>>> 2. Populate from OTel context if available; the exact syntax is
> >>>> implementation-specific;
> >>>> 3. Otherwise, populate from X-Request-Id header if available;
> >>>> 4. Otherwise, don't populate.
> >>>> 5. Create an INDEX on correlation_id.
> >>>>
> >>>> You'll notice that we don't store the ID type, but that doesn't seem
> >>>> necessary.
> >>>>
> >>>> What do you all think?
> >>>>
> >>>> Thanks,
> >>>> Alex
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <dimas@apache.org
> >
> >>>> wrote:
> >>>>>
> >>>>> Hi Yufei,
> >>>>>
> >>>>> [...] tying a BeforeUpdateTable event to the corresponding
> >>>> AfterUpdateTable
> >>>>> event.
> >>>>>
> >>>>>
> >>>>> It looks like this calls for a strong and always present ID in
> >>>> events.This
> >>>>> ID is obviously distincts from OTel context information and should be
> >>>>> persisted separately, IMHO.
> >>>>>
> >>>>> I believe there's still an open question whether the request ID in
> >> the
> >>>>> events should be populated from the x-request-id in REST API headers.
> >>>>>
> >>>>> Yufei: Is that required in your use cases?
> >>>>>
> >>>>> Thanks,
> >>>>> Dmitri.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> I think the first question is whether any downstream use cases
> >> need to
> >>>>>> interpret the semantics of a request ID or OTel context. As I
> >>>> mentioned,
> >>>>>> the original purpose of a request ID was event correlation, e.g.,
> >>>> tying a
> >>>>>> BeforeUpdateTable event to the corresponding AfterUpdateTable
> >> event. If
> >>>>>> that’s the only requirement, an opaque correlation ID is good
> >> enough.
> >>>> We
> >>>>>> could change the field name(request-id) if needed.
> >>>>>>
> >>>>>> If we do have use cases that depend on the meaning of the request
> >> ID or
> >>>>>> OTel context, we can store them in additional_properties (
> >>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%23L134-L134&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3kHvvqJhJTJi61gR6c6CFy
> >>>>>> ).
> >>>>>> This is more flexible because it allows us to:
> >>>>>> 1. Record a key that indicates the format (e.g., x-request-id vs.
> >>>>>> otel-context).
> >>>>>> 2. Preserve both when Polaris receives both headers.
> >>>>>> 3. Add future tracing fields without a schema change.
> >>>>>>
> >>>>>> Yufei
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
> >> dimas@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi All,
> >>>>>>>
> >>>>>>> From my POV the use cases for x-request-id and OTel are slightly
> >>>>>> different.
> >>>>>>> OTel is an observability use case where cross-linking of various
> >>>> pieces
> >>>>>> of
> >>>>>>> data is desired but not required in all cases (data loss is
> >>>> acceptable).
> >>>>>>> The x-request-id use case appears to have some very specific
> >> custom
> >>>>>>> applications without any obvious relationship to OTel.
> >>>>>>>
> >>>>>>> Given the current state of this discussion, I think it makes
> >> sense to
> >>>>>>> handle and persist them separately.
> >>>>>>>
> >>>>>>> However, if we intend x-request-id to act as an observability aid
> >>>>>>> alternative to OTel, then it might be preferable to have a config
> >>>> flag
> >>>>>>> indicating which one of them should be added to persistent event
> >>>> data. In
> >>>>>>> that case we should reuse the same column for them (and possibly
> >> add
> >>>> a
> >>>>>>> "type" prefix, e.g. "x-request-id:ABCDEF" or
> >>>> "traceparent:00-12345...")
> >>>>>>>
> >>>>>>> WDYT?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dmitri.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
> >> adutra@apache.org>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Yufei,
> >>>>>>>>
> >>>>>>>>> I’d suggest persisting a single opaque correlation ID instead
> >>>>>>>>
> >>>>>>>> The problem I see with this is that readers cannot know the
> >> syntax
> >>>> of
> >>>>>>>> the opaque id, and will have to "probe" using some heuristics,
> >>>> e.g. if
> >>>>>>>> the id matches the OTel trace context syntax, asume it's OTel,
> >>>>>>>> otherwise... assume it's something else.
> >>>>>>>>
> >>>>>>>> That's not unfeasible, but I wonder if it isn't better to store
> >>>> the 2
> >>>>>>>> ids separately.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Alex
> >>>>>>>>
> >>>>>>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
> >>>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Thanks Alex for raising this discussion. Thanks everyone for
> >>>> chiming
> >>>>>>> in.
> >>>>>>>>> The plan looks good to me overall.
> >>>>>>>>>
> >>>>>>>>>> Update PolarisEvent, expose request ID if available, expose
> >>>> OTel
> >>>>>>>> context
> >>>>>>>>> if available.
> >>>>>>>>>
> >>>>>>>>>
> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762450034000000&usg=AOvVaw33p8lOR4oYEJqcbo4Sm74f
> persists both
> >>>> request id
> >>>>>>> and
> >>>>>>>>> otel context.
> >>>>>>>>>
> >>>>>>>>> Do we actually need to persist both?
> >>>>>>>>> From the original discussion, the intent of a request ID was
> >> to
> >>>>>>> correlate
> >>>>>>>>> multiple events generated by the same request, for example,
> >>>> linking a
> >>>>>>>>> BeforeUpdateTable and an AfterUpdateTable event. In that
> >> sense,
> >>>> what
> >>>>>> we
> >>>>>>>>> really need is a single opaque identifier that ties related
> >>>> events
> >>>>>>>>> together, regardless of whether it comes from a request ID
> >> or an
> >>>> OTel
> >>>>>>>>> context.
> >>>>>>>>>
> >>>>>>>>> If we persist both fields (requestId and otelContext), it can
> >>>> lead to
> >>>>>>>>> consumer confusion, since downstream systems would need to
> >> decide
> >>>>>> which
> >>>>>>>> one
> >>>>>>>>> to rely on, potentially causing inconsistent event
> >> correlation.
> >>>>>> Unless
> >>>>>>>>> there are concrete use cases where event consumers need
> >> both, I’d
> >>>>>>> suggest
> >>>>>>>>> persisting a single opaque correlation ID instead. This
> >> keeps the
> >>>>>>> schema
> >>>>>>>>> simpler and ensures consistent event linking.
> >>>>>>>>>
> >>>>>>>>> Yufei
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
> >>>> adutra@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>> Here is the PR:
> >>
> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762450034000000&usg=AOvVaw33p8lOR4oYEJqcbo4Sm74f
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Alex
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> >>>>>>> jb@nanthrax.net>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi
> >>>>>>>>>>>
> >>>>>>>>>>> Yeah it looks good to me and I agree we have a consensus.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks !
> >>>>>>>>>>> Regards
> >>>>>>>>>>> JB
> >>>>>>>>>>>
> >>>>>>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
> >>>> adutra@apache.org
> >>>>>>>
> >>>>>>> a
> >>>>>>>>>> écrit :
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the good discussion so far. I think we have
> >> a
> >>>> broad
> >>>>>>>>>> consensus.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Unless anybody disagrees I am going to move forward
> >> with
> >>>> the
> >>>>>> the
> >>>>>>>>>>>> following revised plan, incorporating the latest
> >> feedback:
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. Restore the original functionality for request IDs,
> >>>> change
> >>>>>> the
> >>>>>>>>>>>> default header name back to x-request-id
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 3. Update PolarisEvent, expose request ID if available,
> >>>> expose
> >>>>>>> OTel
> >>>>>>>>>>>> context if available.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 4. Update events table SQL schema: insert request ID if
> >>>>>>> available,
> >>>>>>>>>>>> insert OTel context if available.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Alex
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> >>>>>>>> jb@nanthrax.net
> >>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think it's better to use w3c_trace_context (better
> >>>> than two
> >>>>>>>>>> fields).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regarding x-request-id, I think it's for a different
> >>>> purpose,
> >>>>>>>>>>>>> specifically for persistence. Personally, I think we
> >> can
> >>>>>>> achieve
> >>>>>>>> the
> >>>>>>>>>>>>> same with span and events.
> >>>>>>>>>>>>> If we want to "distinguish" the two layers, it makes
> >>>> sense to
> >>>>>>>> have
> >>>>>>>>>>>>> x-request-id (not sure it's super helpful). Else, I
> >>>> think we
> >>>>>>> can
> >>>>>>>>>>>>> achieve the same with span/event.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Just my $0.01
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards
> >>>>>>>>>>>>> JB
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> >>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested
> >>>> changes as
> >>>>>>>> well.
> >>>>>>>>>> Great
> >>>>>>>>>>>> find, Dmitri!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I’m still debating with myself on whether we need
> >> to
> >>>> store
> >>>>>>> the
> >>>>>>>>>>>> `x-request-id` field as part of the Events persistence.
> >>>> Can we
> >>>>>>>> think
> >>>>>>>>>> of a
> >>>>>>>>>>>> good use case where this is more helpful to the user
> >> than
> >>>> the
> >>>>>>> OTel
> >>>>>>>>>>>> Trace/Span IDs? I am making the assumption here that
> >> those
> >>>> are
> >>>>>>>> still
> >>>>>>>>>> being
> >>>>>>>>>>>> returned back to the client.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>> Adnan Hemani
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> >>>>>>>> adutra@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Dmitri,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yes, I think your suggestion to use just one
> >> field
> >>>>>>>>>> w3c_trace_context
> >>>>>>>>>>>>>>> makes more sense than two fields (span_id and
> >>>> trace_id).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> With that, I also think that we are slowly
> >> drifting
> >>>> into
> >>>>>>>>>>>>>>> implementation considerations; let's get
> >> consensus
> >>>> on the
> >>>>>>>> general
> >>>>>>>>>>>>>>> design first, and we can certainly fine-tune the
> >>>> actual
> >>>>>>> Java
> >>>>>>>>>> methods
> >>>>>>>>>>>>>>> and SQL table columns in the future PR. WDYT?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri
> >> Bourlatchkov <
> >>>>>>>>>>>> dimas@apache.org> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Many thanks for the background info, Adnan!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1 to action items proposed by Alex!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Re: (3) Can we abstract request/trace info into
> >> a
> >>>>>> separate
> >>>>>>>>>> object
> >>>>>>>>>>>> without
> >>>>>>>>>>>>>>>> exposing those accessors on the Event class
> >>>> directly?
> >>>>>> OTel
> >>>>>>>>>> defines
> >>>>>>>>>>>>>>>> trace/span concepts, but in request ID is a bit
> >>>> foreign
> >>>>>> to
> >>>>>>>> OTel.
> >>>>>>>>>>>> Having
> >>>>>>>>>>>>>>>> tracing / request ID isolated in java could help
> >>>> with
> >>>>>>>>>> maintaining
> >>>>>>>>>>>> it and
> >>>>>>>>>>>>>>>> potentially supporting other (custom) tracing
> >>>> methods.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Re: (4) I'd like to propose storing OTel
> >> correlation
> >>>>>> data
> >>>>>>>> in the
> >>>>>>>>>>>> form of a
> >>>>>>>>>>>>>>>> standard context propagation string, e.g. W3C
> >>>>>>> trace-context
> >>>>>>>> [1]
> >>>>>>>>>>>> (same value
> >>>>>>>>>>>>>>>> as its HTTP header), so the column could be
> >> called
> >>>>>>>>>>>> w3c_trace_context or
> >>>>>>>>>>>>>>>> simply trace_context.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Open question: do we need to write a separate,
> >>>>>> individual
> >>>>>>>> trace
> >>>>>>>>>> ID
> >>>>>>>>>>>> field in
> >>>>>>>>>>>>>>>> SQL? I suppose it is not very useful since
> >>>> correlating
> >>>>>> it
> >>>>>>> to
> >>>>>>>>>> other
> >>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>> data already requires understanding OTel context
> >>>>>>> propagation
> >>>>>>>>>> and a
> >>>>>>>>>>>> query
> >>>>>>>>>>>>>>>> against trace_context can still be made using
> >>>>>>>> string-matching
> >>>>>>>>>>>> clauses. We
> >>>>>>>>>>>>>>>> could probably (additionally) store it in the
> >>>> request_id
> >>>>>>>> column
> >>>>>>>>>> if
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> Polaris-specific request ID header is not set.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> As for span ID, I do not really see a use case
> >> for
> >>>>>>>> persisting it
> >>>>>>>>>>>>>>>> individually. It is very specific to OTel trace
> >> data
> >>>>>>>>>> construction.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Actually, using the W3C trace context [1]
> >> encoding
> >>>>>>> probably
> >>>>>>>>>> makes
> >>>>>>>>>>>> sense in
> >>>>>>>>>>>>>>>> the java event representation too. Interested
> >>>> callers
> >>>>>> can
> >>>>>>>> easily
> >>>>>>>>>>>> decode
> >>>>>>>>>>>>>>>> this information since the format is
> >> well-defined.
> >>>> As a
> >>>>>>> side
> >>>>>>>>>>>> benefit, this
> >>>>>>>>>>>>>>>> opens opportunities for downstream event
> >> consumers
> >>>> to
> >>>>>>>> connect
> >>>>>>>>>>>> (propagate
> >>>>>>>>>>>>>>>> context) to traces that produced events based
> >> on the
> >>>>>> event
> >>>>>>>> data
> >>>>>>>>>>>> itself,
> >>>>>>>>>>>>>>>> without relying on the intermediate frameworks.
> >>>> This may
> >>>>>>> be
> >>>>>>>>>>>> desirable since
> >>>>>>>>>>>>>>>> the current Polaris event persistence impl.
> >> writes
> >>>>>> events
> >>>>>>> in
> >>>>>>>>>>>> batches, so
> >>>>>>>>>>>>>>>> the association to individual requests that
> >> produced
> >>>>>> those
> >>>>>>>>>> events
> >>>>>>>>>>>> is lost.
> >>>>>>>>>>>>>>>> Whether to perform this kind of context
> >> propagation
> >>>> will
> >>>>>>> be
> >>>>>>>> at
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> discretion of the event consumer, of course
> >>>> (outside of
> >>>>>>>> Polaris
> >>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context/%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2fn0lRsTx-f9r4PCz8wmJK&source=gmail-imap&ust=1762450034000000&usg=AOvVaw0JHBUVrCE7tookVBiFjNdP
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
> >> <
> >>>>>>>>>> adutra@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thank you for chiming in; the context around
> >>>> request
> >>>>>> IDs
> >>>>>>>> is now
> >>>>>>>>>>>> clear.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Trying to summarize this thread into actionable
> >>>> items,
> >>>>>>>> here's
> >>>>>>>>>> what
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> propose:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1. Restore the original functionality for
> >> request
> >>>> IDs.
> >>>>>>>>>>>>>>>>> - Change the default header name back to
> >>>>>> x-request-id
> >>>>>>>>>> (despite
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> x- prefix being deprecated), but keep it
> >>>> configurable
> >>>>>> as
> >>>>>>>> today.
> >>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related
> >>>> functionality.
> >>>>>>>>>>>>>>>>> - Do not generate a request ID if the REST
> >>>> client
> >>>>>>>> doesn't
> >>>>>>>>>>>> provide one.
> >>>>>>>>>>>>>>>>> 3. Update PolarisEvent:
> >>>>>>>>>>>>>>>>> - Expose new requestId(), traceId(),
> >> spanId()
> >>>>>> methods,
> >>>>>>>> all
> >>>>>>>>>>>> nullable.
> >>>>>>>>>>>>>>>>> - This would align with the emerging
> >> consensus
> >>>>>> around
> >>>>>>>>>> including
> >>>>>>>>>>>>>>>>> contextual information in PolarisEvent [1].
> >>>>>>>>>>>>>>>>> 4. Update events table SQL schema:
> >>>>>>>>>>>>>>>>> - Insert the client-provided request ID
> >> into the
> >>>>>>>> request_id
> >>>>>>>>>>>>>>>>> column; otherwise, insert null.
> >>>>>>>>>>>>>>>>> - Add two new nullable columns, trace_id and
> >>>>>> span_id,
> >>>>>>>> and
> >>>>>>>>>>>> populate
> >>>>>>>>>>>>>>>>> them if OTel is enabled.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> From our discussions, I think it's important
> >> to not
> >>>>>>>> conflate
> >>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>> tracing fields with Envoy tracing fields,
> >> which is
> >>>> why
> >>>>>> I
> >>>>>>>>>> suggest we
> >>>>>>>>>>>>>>>>> use separate fields / columns for them.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Would the above plan work for everyone?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> [1]:
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw02wPvb0qxRzYAKEP0h8l9T&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3AIjY12CwTp7ItIvzTOIIN
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> >>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks to Alex for starting this thread -
> >> because
> >>>> of
> >>>>>>>> this, I’m
> >>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> coming to the realization that OTel Trace and
> >> Span
> >>>> IDs
> >>>>>>> are
> >>>>>>>>>> coming
> >>>>>>>>>>>> built-in
> >>>>>>>>>>>>>>>>> with Quarkus and my previous work to generate a
> >>>>>>>>>>>> Request/Correlation ID is
> >>>>>>>>>>>>>>>>> likely not needed as a result. My original
> >>>> motivation
> >>>>>> for
> >>>>>>>>>>>> generation of a
> >>>>>>>>>>>>>>>>> Request/Correlation ID was to ensure that any
> >>>> client
> >>>>>> can
> >>>>>>>>>> uniquely
> >>>>>>>>>>>> identify
> >>>>>>>>>>>>>>>>> a request made to Polaris, which would be
> >>>> especially
> >>>>>>>> useful for
> >>>>>>>>>>>> debugging
> >>>>>>>>>>>>>>>>> failing requests or identifying call patterns.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we
> >>>> should
> >>>>>>>> remove
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Request/Correlation ID generation and always
> >> use
> >>>> the
> >>>>>> OTel
> >>>>>>>>>>>> trace/span IDs
> >>>>>>>>>>>>>>>>> (which come for free with Quarkus) instead for
> >> the
> >>>>>>>> Correlation
> >>>>>>>>>> ID
> >>>>>>>>>>>> unless a
> >>>>>>>>>>>>>>>>> valid header is present, which would take over
> >> as
> >>>> the
> >>>>>>>>>> Correlation
> >>>>>>>>>>>> ID
> >>>>>>>>>>>>>>>>> instead.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To answer Dmitri’s question re Polaris
> >> Events: The
> >>>>>>>> intended
> >>>>>>>>>> use
> >>>>>>>>>>>> case is
> >>>>>>>>>>>>>>>>> to provide some sort of correlation between
> >> events
> >>>> that
> >>>>>>>> have
> >>>>>>>>>>>> occurred as
> >>>>>>>>>>>>>>>>> part of the same request. For example, if a
> >> user
> >>>> makes
> >>>>>> an
> >>>>>>>>>>>> CommitTransaction
> >>>>>>>>>>>>>>>>> request, it would be helpful to see all
> >> UpdateTable
> >>>>>> calls
> >>>>>>>> that
> >>>>>>>>>>>> were made as
> >>>>>>>>>>>>>>>>> part of that one user request.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>> Adnan Hemani
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
> >>>> Bourlatchkov <
> >>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Michael,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Just to confirm: if / when we restore that,
> >>>> Polaris
> >>>>>>> will
> >>>>>>>> NOT
> >>>>>>>>>>>> generate
> >>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>> IDs in case the header is not present in the
> >>>> request,
> >>>>>>>>>> correct?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I believe x-request-id can co-exist with
> >> OTel.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> What about adding request IDs to events
> >> [1][2]?
> >>>>>> What's
> >>>>>>>> the
> >>>>>>>>>>>> intended use
> >>>>>>>>>>>>>>>>>>> case for that? Could you share some context
> >> here
> >>>> too?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event
> >>>>>> persistence
> >>>>>>>> as
> >>>>>>>>>>>> "beta" in
> >>>>>>>>>>>>>>>>>>> 1.2.0... This discussion adds another point
> >>>> towards
> >>>>>>>> that, I
> >>>>>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%252523L97%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Pioys4ROm8vYM_mdx4ygH&source=gmail-imap&ust=1762450034000000&usg=AOvVaw39NasMg4Tgyrfe5Bj_U4xY
> >>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%252523L66%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw12-7e3ahm2sLSkLSNqTecm%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1PGW6uFwtUN8F1dOlJwrpm&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3y_SObfpEJH_K_8RD9lVTL
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [2877]
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2877%252523discussion_r2456300613%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2_e3B3qU8kJxdMmQx-q6bZ
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
> >> Collado <
> >>>>>>>>>>>>>>>>> collado.mike@gmail.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hey Dmitri
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The generating a request id is new code
> >> that was
> >>>>>> added
> >>>>>>>>>> after the
> >>>>>>>>>>>>>>>>> original
> >>>>>>>>>>>>>>>>>>>> x-request-id support. You can see the state
> >>>> from ~1
> >>>>>>> year
> >>>>>>>>>> ago, we
> >>>>>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>>>>>>> request_id as the header we used for the
> >> MDC -
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%252523L415-L416%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw34DksOw7DSHJu8PfQQ5bT6&source=gmail-imap&ust=1762450034000000&usg=AOvVaw1g09FwhrTquJs9utV9CAX9
> >>>>>>>>>>>>>>>>>>>> . At some point, it was changed to be
> >>>> configurable,
> >>>>>>>> then the
> >>>>>>>>>>>>>>>>>>>> ContextResolverFilter filter was
> >>>>>> refactored/eliminated
> >>>>>>>> and
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but
> >>>> lost
> >>>>>> some
> >>>>>>>> of
> >>>>>>>>>> its
> >>>>>>>>>>>> original
> >>>>>>>>>>>>>>>>>>>> functionality.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> My personal opinion is that restoring
> >> support
> >>>> for
> >>>>>> the
> >>>>>>>>>>>> x-request-id
> >>>>>>>>>>>>>>>>> header
> >>>>>>>>>>>>>>>>>>>> is something that we should do, but if the
> >>>> header
> >>>>>>> isn't
> >>>>>>>>>> present,
> >>>>>>>>>>>>>>>>> falling
> >>>>>>>>>>>>>>>>>>>> back on simply using OTel trace ids is good
> >>>> enough
> >>>>>>>> (better,
> >>>>>>>>>>>> even) than
> >>>>>>>>>>>>>>>>>>>> generating another random request id.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> >>>>>> Bourlatchkov <
> >>>>>>>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi Michael,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for the info!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes
> >>>> sense to
> >>>>>>> me.
> >>>>>>>>>>>> However, I
> >>>>>>>>>>>>>>>>>>>> wonder:
> >>>>>>>>>>>>>>>>>>>>> why would Polaris need to generate a new
> >>>> request ID
> >>>>>>>> inside
> >>>>>>>>>> its
> >>>>>>>>>>>>>>>>> code?..
> >>>>>>>>>>>>>>>>>>>>> and return it in response headers?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> How important is it to propagate this ID to
> >>>> Polaris
> >>>>>>>> Events?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'm just trying to understand the full
> >> context
> >>>> :)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
> >>>> Collado <
> >>>>>>>>>>>>>>>>> collado.mike@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I think the original intention for this
> >>>> requestId
> >>>>>>>> field
> >>>>>>>>>> was to
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> request propagation from load balancers,
> >> like
> >>>>>> Envoy
> >>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw3LNqtilZ2OoJfE7yEwraZa&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2hTMXu2DU30Lg_5UBzsDJw
> >>>>>>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask
> >> me
> >>>> why
> >>>>>> the
> >>>>>>>>>> default
> >>>>>>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was
> >>>> originally
> >>>>>> a
> >>>>>>>> custom
> >>>>>>>>>>>> thing,
> >>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> we changed to integrate with existing
> >>>> conventions.
> >>>>>>>>>>>> Unfortunately,
> >>>>>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>> through the code, I think that the actual
> >>>>>> functional
> >>>>>>>>>> plumbing
> >>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>> lost
> >>>>>>>>>>>>>>>>>>>>>> in the course of multiple refactors
> >> around the
> >>>>>> call
> >>>>>>>>>> context
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> resolver. I
> >>>>>>>>>>>>>>>>>>>>>> don't see references to that property or
> >> the
> >>>>>>>> underlying
> >>>>>>>>>>>> header.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id
> >> header
> >>>>>> feels
> >>>>>>>> like
> >>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> should definitely keep, especially when
> >>>> Polaris is
> >>>>>>> one
> >>>>>>>>>>>> service in a
> >>>>>>>>>>>>>>>>>>>> mesh
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> services that maybe don't have OTel
> >>>> integration.
> >>>>>>> I'm a
> >>>>>>>>>> fan of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>> standard, but it's not entirely
> >> ubiquitous and
> >>>>>> there
> >>>>>>>> are
> >>>>>>>>>> many
> >>>>>>>>>>>>>>>>>>>> middleware
> >>>>>>>>>>>>>>>>>>>>>> layers that know how to forward on the
> >>>>>> x-request-id
> >>>>>>>>>> header.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
> >> Stupp <
> >>>>>>>>>> snazy@snazy.de>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability
> >> with
> >>>> the
> >>>>>>>> existing
> >>>>>>>>>>>> de-facto
> >>>>>>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users
> >> to
> >>>>>>> integrate
> >>>>>>>>>> into
> >>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>> observability platforms.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> >>>>>>> Bourlatchkov <
> >>>>>>>>>>>>>>>>>>>> dimas@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Alex and All,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I certainly support the idea of
> >> following
> >>>> OTel
> >>>>>>>>>> standards for
> >>>>>>>>>>>>>>>>>>>>> achieving
> >>>>>>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests
> >> and/or
> >>>>>> events.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> As to what form the correlation data
> >> should
> >>>>>> take,
> >>>>>>> I
> >>>>>>>>>> believe
> >>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context"
> >>>> represents.
> >>>>>>>> So, I
> >>>>>>>>>>>> believe it
> >>>>>>>>>>>>>>>>>>>>> makes
> >>>>>>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard
> >>>> context
> >>>>>>>>>> propagators
> >>>>>>>>>>>> at the
> >>>>>>>>>>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
> >>>>>>>> information,
> >>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>> returning
> >>>>>>>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>>>>> other "correlation" data in the
> >> response is
> >>>>>>>> redundant, I
> >>>>>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If the incoming request does not have
> >> OTel
> >>>>>> context
> >>>>>>>> info,
> >>>>>>>>>>>> what is
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
> >>>>>>>> "correlation
> >>>>>>>>>> ID"?
> >>>>>>>>>>>> How is
> >>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> envisioned to be used?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If the intention is to correlate a
> >> Polaris
> >>>>>>> response
> >>>>>>>>>>>> (operation)
> >>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>> events
> >>>>>>>>>>>>>>>>>>>>>>>> that might have resulted from its
> >>>> execution, I
> >>>>>>>> believe a
> >>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>> robust
> >>>>>>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel
> >>>> trace
> >>>>>> info
> >>>>>>>>>>>> (starting a new
> >>>>>>>>>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>> necessary) into event data. Then,
> >> Polaris
> >>>> can
> >>>>>> also
> >>>>>>>>>> return
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>>> context
> >>>>>>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a
> >> bit
> >>>>>> awkward
> >>>>>>>> from
> >>>>>>>>>> the
> >>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>>>> perspective, but might be an option for
> >>>>>> supporting
> >>>>>>>>>> custom
> >>>>>>>>>>>>>>>>>>>>> correlators.
> >>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The
> >>>> header
> >>>>>>> name
> >>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace
> >> Context.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to
> >>>> extract
> >>>>>>> the
> >>>>>>>>>> trace
> >>>>>>>>>>>> ID from
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> response and from events and find
> >> related
> >>>> data.
> >>>>>>>>>> Granted, it
> >>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> require
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to
> >>>> decode
> >>>>>>> trace
> >>>>>>>> IDs
> >>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>> OTel
> >>>>>>>>>>>>>>>>>>>>>>>> context, but the format is standard and
> >> not
> >>>>>>>> complex. The
> >>>>>>>>>>>> benefit
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily
> >>>> integrate
> >>>>>>>> with
> >>>>>>>>>>>>>>>>>>>> OTel-compatible
> >>>>>>>>>>>>>>>>>>>>>>>> observability platforms regardless of
> >>>> whether
> >>>>>> any
> >>>>>>>>>> particular
> >>>>>>>>>>>>>>>>>>>>> deployment
> >>>>>>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
> >> Alexandre
> >>>> Dutra
> >>>>>> <
> >>>>>>>>>>>>>>>>> adutra@apache.org
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of
> >> "request
> >>>> ID",
> >>>>>>> but
> >>>>>>>> its
> >>>>>>>>>>>> purpose is
> >>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
> >>>>>>>> observability
> >>>>>>>>>>>> feature to
> >>>>>>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR
> >> aims
> >>>> to
> >>>>>>>> rename it
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> "correlation
> >>>>>>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry
> >>>>>> standards
> >>>>>>>> [1].
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light
> >>>> overlaps
> >>>>>>> with
> >>>>>>>>>> core
> >>>>>>>>>>>>>>>>>>>> telemetry
> >>>>>>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is
> >>>> enabled
> >>>>>> in
> >>>>>>>>>> Polaris,
> >>>>>>>>>>>> each
> >>>>>>>>>>>>>>>>>>>>>>>>> request already has a trace ID and
> >> span ID,
> >>>>>>> making
> >>>>>>>> a
> >>>>>>>>>>>> separate
> >>>>>>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and
> >> span
> >>>> ID
> >>>>>> in
> >>>>>>>>>> Polaris
> >>>>>>>>>>>> events,
> >>>>>>>>>>>>>>>>>>>>>>>>> rather than the generated correlation
> >> ID,
> >>>> would
> >>>>>>>>>>>> significantly
> >>>>>>>>>>>>>>>>>>>>>> simplify
> >>>>>>>>>>>>>>>>>>>>>>>>> correlation of events with other
> >> traces.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following
> >> changes:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace
> >> ID and
> >>>>>> span
> >>>>>>>> ID as
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>>>>>>> ID for the request, instead of
> >> generating a
> >>>>>>> random
> >>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>> ID.
> >>>>>>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> >>>>>> correlation
> >>>>>>>> ID
> >>>>>>>>>>>> header is
> >>>>>>>>>>>>>>>>>>>>>> present
> >>>>>>>>>>>>>>>>>>>>>>>>> in the request, use it.
> >>>>>>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions
> >> is
> >>>> met,
> >>>>>>>> generate
> >>>>>>>>>> a
> >>>>>>>>>>>> random
> >>>>>>>>>>>>>>>>>>>>>>>>> correlation ID.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best
> >>>> approach
> >>>>>>> when a
> >>>>>>>>>>>> correlation
> >>>>>>>>>>>>>>>>>>>> ID
> >>>>>>>>>>>>>>>>>>>>>>>>> header is present in the request.
> >> However,
> >>>> I
> >>>>>>>> believe it
> >>>>>>>>>>>> would be
> >>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is
> >>>> enabled, as
> >>>>>>>> OTel
> >>>>>>>>>>>> offers a
> >>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server
> >> trace
> >>>>>>>> propagation,
> >>>>>>>>>>>> e.g. W3C
> >>>>>>>>>>>>>>>>>>>>>> Trace
> >>>>>>>>>>>>>>>>>>>>>>>>> Context propagation [2].
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> Alex
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> [1]:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2757%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Oe-25vtt4gLVSMFMtSVNg&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2ohYKOQ8XU5-GBlDo6zYGC
> >>>>>>>>>>>>>>>>>>>>>>>>> [2]:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1TRdkzABc_7U-_KZ1MZ59v&source=gmail-imap&ust=1762450034000000&usg=AOvVaw0cUBLtXCkRoH5pL57naUhY
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
>
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi folks,
Unfortunately, I don’t agree on most of the discussion past Alex’s re-summarization. I do, however, agree on the requirements stated by Alex in that email.
Here’s what I believe we should be doing in summary:
1. Keep the existing `request_id` column.
* To be fair, I don’t fully understand the reason why we are trying to change a ubiquitous concept such as `request_id` to something named as `correlation_id`. Are we trying to state explicitly that the correlation ID and request ID should/are not be the same? If someone can please explain this part, that would be helpful.
2. In the `request_id` field, use the `X-Request-Id` header value if it exists - as that is explicitly what the client is asking for and will mark as the request ID in their logs.
2a. Otherwise, use the OTel Trace ID as the request ID. This should always exist, as it is provided by Quarkus on a per-request basis.
2b. If, in the future, we are not provided an OTel Trace ID (for whatever reason), one should be generated.
4. Keep the `request_id` as part of the response headers.
* This allows clients to note down what the request ID that was used and/or generated. Without it, how will clients who don’t provide a `X-Request-Id` header be able to communicate which request they are referring to if they need to debug something?
5. I’m unsure on indexing over request_id or keeping the request_id_type. I don’t see these as a hard use case today, but we can always add it as an improvement in the future.
To put this all another way (perhaps more philosophically, but the way I am thinking about things in general):
* There should be exactly 1 request ID that is used when a client makes a request to Polaris. That request ID can be dictated by the client (which I still find a bit odd when looking at web server architecture in general, but I understand the use case we have for it) or generated by the server.
* That request ID should then be used in any persisting anything that involve request identification (in this particular case, for Events, but not limited to if there’s something else in the future.)
* That request ID should be made aware of to the client, so it knows how to identify the request in the case of future debugging.
Please let me know your thoughts.
Best,
Adnan Hemani
> On Oct 30, 2025, at 10:26 AM, Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Alex,
>
> SGTM (both not using request ID _response_ headers and deferring type info
> in correlation ID values).
>
> Cheers,
> Dmitri.
>
> On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org> wrote:
>
>> Hi Dmitri,
>>
>>> I believe we still need to clarify / confirm whether returning any
>> request / correlation ID in _response_ headers is required.
>>
>> Indeed, that specific item was still pending. FYI I'm going to
>> *remove* the echoing of the request ID in the response headers, unless
>> someone objects.
>>
>>> it could be useful to embed some type data into the value to help with
>> interpretation
>>
>> I'm not sure we need this at this moment. I'd suggest that we leave
>> this for further improvements in order to not over-specify the
>> requirements. Given that the events feature is beta, this shouldn't be
>> a problem.
>>
>> Thanks,
>> Alex
>>
>>
>> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
>> wrote:
>>>
>>> Thanks for the nice summary, Alex!
>>>
>>> To All:
>>>
>>> I'm not sure an INDEX on correlation_id is that useful... Polaris itself
>>> does not use that index, right? Do we expect end users to query events
>>> directly from the database? This will work only with RDBMS, but probably
>>> not with the proposed NoSQL persistence.
>>>
>>> I believe we still need to clarify / confirm whether returning any
>> request
>>> / correlation ID in _response_ headers is required.
>>>
>>> From my POV, I do not see a use case for it, but I wonder what other
>> people
>>> think / need.
>>>
>>> As for persisting correlation ID type, I agree that we do not need a
>>> separate column for that. Still, it could be useful to embed some type
>> data
>>> into the value to help with interpretation, specifically for telemetry
>> use
>>> cases (both x-request-id and OTel). For example, we could
>>> persist "traceparent:00-12345-...". Then, the reader would be able to
>>> identify this as information following the W3C spec [1], parse it and
>> link
>>> to other traces (if desired). That processing is optional and at the
>>> discretion of the event data reader, but making the value no-so-opaque
>>> upfront can enable a lot of interesting options for users without much
>>> extra overhead in Polaris. WDYT?
>>>
>>> [1] https://www.google.com/url?q=https://www.w3.org/TR/2021/REC-trace-context-1-20211123/&source=gmail-imap&ust=1762450034000000&usg=AOvVaw1K633xlm1Dj0NjcNcpd41A
>>>
>>> Thanks,
>>> Dmitri.
>>>
>>> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
>> wrote:
>>>
>>>> Hi all,
>>>>
>>>>> we would only persist one or the other, not both.
>>>>
>>>> My apologies, I confess that this wasn't my understanding of the
>> desired
>>>> design.
>>>>
>>>> Let's try to gather the requirements again:
>>>>
>>>> 1. Only one correlation ID is enough
>>>> 2. The correlation ID is an opaque string
>>>> 3. The main use case is to find events with matching correlation IDs
>>>> 4. The only query pattern is by exact match
>>>>
>>>> Regarding *where* to store the correlation ID, and expanding on what
>>>> Dmitri said: the requirement to find matching records by correlation
>>>> ID clearly suggests a separate column, and with an INDEX on it.
>>>> Storing it in additional_properties will be inefficient.
>>>>
>>>> If we all agree on the above, here is what I propose:
>>>>
>>>> 1. Rename the existing column request_id to correlation_id;
>>>> 2. Populate from OTel context if available; the exact syntax is
>>>> implementation-specific;
>>>> 3. Otherwise, populate from X-Request-Id header if available;
>>>> 4. Otherwise, don't populate.
>>>> 5. Create an INDEX on correlation_id.
>>>>
>>>> You'll notice that we don't store the ID type, but that doesn't seem
>>>> necessary.
>>>>
>>>> What do you all think?
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <di...@apache.org>
>>>> wrote:
>>>>>
>>>>> Hi Yufei,
>>>>>
>>>>> [...] tying a BeforeUpdateTable event to the corresponding
>>>> AfterUpdateTable
>>>>> event.
>>>>>
>>>>>
>>>>> It looks like this calls for a strong and always present ID in
>>>> events.This
>>>>> ID is obviously distincts from OTel context information and should be
>>>>> persisted separately, IMHO.
>>>>>
>>>>> I believe there's still an open question whether the request ID in
>> the
>>>>> events should be populated from the x-request-id in REST API headers.
>>>>>
>>>>> Yufei: Is that required in your use cases?
>>>>>
>>>>> Thanks,
>>>>> Dmitri.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
>> wrote:
>>>>>
>>>>>> I think the first question is whether any downstream use cases
>> need to
>>>>>> interpret the semantics of a request ID or OTel context. As I
>>>> mentioned,
>>>>>> the original purpose of a request ID was event correlation, e.g.,
>>>> tying a
>>>>>> BeforeUpdateTable event to the corresponding AfterUpdateTable
>> event. If
>>>>>> that’s the only requirement, an opaque correlation ID is good
>> enough.
>>>> We
>>>>>> could change the field name(request-id) if needed.
>>>>>>
>>>>>> If we do have use cases that depend on the meaning of the request
>> ID or
>>>>>> OTel context, we can store them in additional_properties (
>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%23L134-L134&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3kHvvqJhJTJi61gR6c6CFy
>>>>>> ).
>>>>>> This is more flexible because it allows us to:
>>>>>> 1. Record a key that indicates the format (e.g., x-request-id vs.
>>>>>> otel-context).
>>>>>> 2. Preserve both when Polaris receives both headers.
>>>>>> 3. Add future tracing fields without a schema change.
>>>>>>
>>>>>> Yufei
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
>> dimas@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> From my POV the use cases for x-request-id and OTel are slightly
>>>>>> different.
>>>>>>> OTel is an observability use case where cross-linking of various
>>>> pieces
>>>>>> of
>>>>>>> data is desired but not required in all cases (data loss is
>>>> acceptable).
>>>>>>> The x-request-id use case appears to have some very specific
>> custom
>>>>>>> applications without any obvious relationship to OTel.
>>>>>>>
>>>>>>> Given the current state of this discussion, I think it makes
>> sense to
>>>>>>> handle and persist them separately.
>>>>>>>
>>>>>>> However, if we intend x-request-id to act as an observability aid
>>>>>>> alternative to OTel, then it might be preferable to have a config
>>>> flag
>>>>>>> indicating which one of them should be added to persistent event
>>>> data. In
>>>>>>> that case we should reuse the same column for them (and possibly
>> add
>>>> a
>>>>>>> "type" prefix, e.g. "x-request-id:ABCDEF" or
>>>> "traceparent:00-12345...")
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dmitri.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
>> adutra@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Yufei,
>>>>>>>>
>>>>>>>>> I’d suggest persisting a single opaque correlation ID instead
>>>>>>>>
>>>>>>>> The problem I see with this is that readers cannot know the
>> syntax
>>>> of
>>>>>>>> the opaque id, and will have to "probe" using some heuristics,
>>>> e.g. if
>>>>>>>> the id matches the OTel trace context syntax, asume it's OTel,
>>>>>>>> otherwise... assume it's something else.
>>>>>>>>
>>>>>>>> That's not unfeasible, but I wonder if it isn't better to store
>>>> the 2
>>>>>>>> ids separately.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
>>>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks Alex for raising this discussion. Thanks everyone for
>>>> chiming
>>>>>>> in.
>>>>>>>>> The plan looks good to me overall.
>>>>>>>>>
>>>>>>>>>> Update PolarisEvent, expose request ID if available, expose
>>>> OTel
>>>>>>>> context
>>>>>>>>> if available.
>>>>>>>>>
>>>>>>>>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762450034000000&usg=AOvVaw33p8lOR4oYEJqcbo4Sm74f persists both
>>>> request id
>>>>>>> and
>>>>>>>>> otel context.
>>>>>>>>>
>>>>>>>>> Do we actually need to persist both?
>>>>>>>>> From the original discussion, the intent of a request ID was
>> to
>>>>>>> correlate
>>>>>>>>> multiple events generated by the same request, for example,
>>>> linking a
>>>>>>>>> BeforeUpdateTable and an AfterUpdateTable event. In that
>> sense,
>>>> what
>>>>>> we
>>>>>>>>> really need is a single opaque identifier that ties related
>>>> events
>>>>>>>>> together, regardless of whether it comes from a request ID
>> or an
>>>> OTel
>>>>>>>>> context.
>>>>>>>>>
>>>>>>>>> If we persist both fields (requestId and otelContext), it can
>>>> lead to
>>>>>>>>> consumer confusion, since downstream systems would need to
>> decide
>>>>>> which
>>>>>>>> one
>>>>>>>>> to rely on, potentially causing inconsistent event
>> correlation.
>>>>>> Unless
>>>>>>>>> there are concrete use cases where event consumers need
>> both, I’d
>>>>>>> suggest
>>>>>>>>> persisting a single opaque correlation ID instead. This
>> keeps the
>>>>>>> schema
>>>>>>>>> simpler and ensures consistent event linking.
>>>>>>>>>
>>>>>>>>> Yufei
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
>>>> adutra@apache.org>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> Here is the PR:
>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762450034000000&usg=AOvVaw33p8lOR4oYEJqcbo4Sm74f
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
>>>>>>> jb@nanthrax.net>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> Yeah it looks good to me and I agree we have a consensus.
>>>>>>>>>>>
>>>>>>>>>>> Thanks !
>>>>>>>>>>> Regards
>>>>>>>>>>> JB
>>>>>>>>>>>
>>>>>>>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
>>>> adutra@apache.org
>>>>>>>
>>>>>>> a
>>>>>>>>>> écrit :
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the good discussion so far. I think we have
>> a
>>>> broad
>>>>>>>>>> consensus.
>>>>>>>>>>>>
>>>>>>>>>>>> Unless anybody disagrees I am going to move forward
>> with
>>>> the
>>>>>> the
>>>>>>>>>>>> following revised plan, incorporating the latest
>> feedback:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Restore the original functionality for request IDs,
>>>> change
>>>>>> the
>>>>>>>>>>>> default header name back to x-request-id
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. Update PolarisEvent, expose request ID if available,
>>>> expose
>>>>>>> OTel
>>>>>>>>>>>> context if available.
>>>>>>>>>>>>
>>>>>>>>>>>> 4. Update events table SQL schema: insert request ID if
>>>>>>> available,
>>>>>>>>>>>> insert OTel context if available.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Alex
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
>>>>>>>> jb@nanthrax.net
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think it's better to use w3c_trace_context (better
>>>> than two
>>>>>>>>>> fields).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regarding x-request-id, I think it's for a different
>>>> purpose,
>>>>>>>>>>>>> specifically for persistence. Personally, I think we
>> can
>>>>>>> achieve
>>>>>>>> the
>>>>>>>>>>>>> same with span and events.
>>>>>>>>>>>>> If we want to "distinguish" the two layers, it makes
>>>> sense to
>>>>>>>> have
>>>>>>>>>>>>> x-request-id (not sure it's super helpful). Else, I
>>>> think we
>>>>>>> can
>>>>>>>>>>>>> achieve the same with span/event.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just my $0.01
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> JB
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested
>>>> changes as
>>>>>>>> well.
>>>>>>>>>> Great
>>>>>>>>>>>> find, Dmitri!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I’m still debating with myself on whether we need
>> to
>>>> store
>>>>>>> the
>>>>>>>>>>>> `x-request-id` field as part of the Events persistence.
>>>> Can we
>>>>>>>> think
>>>>>>>>>> of a
>>>>>>>>>>>> good use case where this is more helpful to the user
>> than
>>>> the
>>>>>>> OTel
>>>>>>>>>>>> Trace/Span IDs? I am making the assumption here that
>> those
>>>> are
>>>>>>>> still
>>>>>>>>>> being
>>>>>>>>>>>> returned back to the client.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
>>>>>>>> adutra@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Dmitri,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, I think your suggestion to use just one
>> field
>>>>>>>>>> w3c_trace_context
>>>>>>>>>>>>>>> makes more sense than two fields (span_id and
>>>> trace_id).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> With that, I also think that we are slowly
>> drifting
>>>> into
>>>>>>>>>>>>>>> implementation considerations; let's get
>> consensus
>>>> on the
>>>>>>>> general
>>>>>>>>>>>>>>> design first, and we can certainly fine-tune the
>>>> actual
>>>>>>> Java
>>>>>>>>>> methods
>>>>>>>>>>>>>>> and SQL table columns in the future PR. WDYT?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri
>> Bourlatchkov <
>>>>>>>>>>>> dimas@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Many thanks for the background info, Adnan!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 to action items proposed by Alex!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Re: (3) Can we abstract request/trace info into
>> a
>>>>>> separate
>>>>>>>>>> object
>>>>>>>>>>>> without
>>>>>>>>>>>>>>>> exposing those accessors on the Event class
>>>> directly?
>>>>>> OTel
>>>>>>>>>> defines
>>>>>>>>>>>>>>>> trace/span concepts, but in request ID is a bit
>>>> foreign
>>>>>> to
>>>>>>>> OTel.
>>>>>>>>>>>> Having
>>>>>>>>>>>>>>>> tracing / request ID isolated in java could help
>>>> with
>>>>>>>>>> maintaining
>>>>>>>>>>>> it and
>>>>>>>>>>>>>>>> potentially supporting other (custom) tracing
>>>> methods.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Re: (4) I'd like to propose storing OTel
>> correlation
>>>>>> data
>>>>>>>> in the
>>>>>>>>>>>> form of a
>>>>>>>>>>>>>>>> standard context propagation string, e.g. W3C
>>>>>>> trace-context
>>>>>>>> [1]
>>>>>>>>>>>> (same value
>>>>>>>>>>>>>>>> as its HTTP header), so the column could be
>> called
>>>>>>>>>>>> w3c_trace_context or
>>>>>>>>>>>>>>>> simply trace_context.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Open question: do we need to write a separate,
>>>>>> individual
>>>>>>>> trace
>>>>>>>>>> ID
>>>>>>>>>>>> field in
>>>>>>>>>>>>>>>> SQL? I suppose it is not very useful since
>>>> correlating
>>>>>> it
>>>>>>> to
>>>>>>>>>> other
>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>> data already requires understanding OTel context
>>>>>>> propagation
>>>>>>>>>> and a
>>>>>>>>>>>> query
>>>>>>>>>>>>>>>> against trace_context can still be made using
>>>>>>>> string-matching
>>>>>>>>>>>> clauses. We
>>>>>>>>>>>>>>>> could probably (additionally) store it in the
>>>> request_id
>>>>>>>> column
>>>>>>>>>> if
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> Polaris-specific request ID header is not set.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As for span ID, I do not really see a use case
>> for
>>>>>>>> persisting it
>>>>>>>>>>>>>>>> individually. It is very specific to OTel trace
>> data
>>>>>>>>>> construction.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Actually, using the W3C trace context [1]
>> encoding
>>>>>>> probably
>>>>>>>>>> makes
>>>>>>>>>>>> sense in
>>>>>>>>>>>>>>>> the java event representation too. Interested
>>>> callers
>>>>>> can
>>>>>>>> easily
>>>>>>>>>>>> decode
>>>>>>>>>>>>>>>> this information since the format is
>> well-defined.
>>>> As a
>>>>>>> side
>>>>>>>>>>>> benefit, this
>>>>>>>>>>>>>>>> opens opportunities for downstream event
>> consumers
>>>> to
>>>>>>>> connect
>>>>>>>>>>>> (propagate
>>>>>>>>>>>>>>>> context) to traces that produced events based
>> on the
>>>>>> event
>>>>>>>> data
>>>>>>>>>>>> itself,
>>>>>>>>>>>>>>>> without relying on the intermediate frameworks.
>>>> This may
>>>>>>> be
>>>>>>>>>>>> desirable since
>>>>>>>>>>>>>>>> the current Polaris event persistence impl.
>> writes
>>>>>> events
>>>>>>> in
>>>>>>>>>>>> batches, so
>>>>>>>>>>>>>>>> the association to individual requests that
>> produced
>>>>>> those
>>>>>>>>>> events
>>>>>>>>>>>> is lost.
>>>>>>>>>>>>>>>> Whether to perform this kind of context
>> propagation
>>>> will
>>>>>>> be
>>>>>>>> at
>>>>>>>>>> the
>>>>>>>>>>>>>>>> discretion of the event consumer, of course
>>>> (outside of
>>>>>>>> Polaris
>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context/%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2fn0lRsTx-f9r4PCz8wmJK&source=gmail-imap&ust=1762450034000000&usg=AOvVaw0JHBUVrCE7tookVBiFjNdP
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
>> <
>>>>>>>>>> adutra@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you for chiming in; the context around
>>>> request
>>>>>> IDs
>>>>>>>> is now
>>>>>>>>>>>> clear.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Trying to summarize this thread into actionable
>>>> items,
>>>>>>>> here's
>>>>>>>>>> what
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> propose:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. Restore the original functionality for
>> request
>>>> IDs.
>>>>>>>>>>>>>>>>> - Change the default header name back to
>>>>>> x-request-id
>>>>>>>>>> (despite
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> x- prefix being deprecated), but keep it
>>>> configurable
>>>>>> as
>>>>>>>> today.
>>>>>>>>>>>>>>>>> 2. Remove RequestIdGenerator and related
>>>> functionality.
>>>>>>>>>>>>>>>>> - Do not generate a request ID if the REST
>>>> client
>>>>>>>> doesn't
>>>>>>>>>>>> provide one.
>>>>>>>>>>>>>>>>> 3. Update PolarisEvent:
>>>>>>>>>>>>>>>>> - Expose new requestId(), traceId(),
>> spanId()
>>>>>> methods,
>>>>>>>> all
>>>>>>>>>>>> nullable.
>>>>>>>>>>>>>>>>> - This would align with the emerging
>> consensus
>>>>>> around
>>>>>>>>>> including
>>>>>>>>>>>>>>>>> contextual information in PolarisEvent [1].
>>>>>>>>>>>>>>>>> 4. Update events table SQL schema:
>>>>>>>>>>>>>>>>> - Insert the client-provided request ID
>> into the
>>>>>>>> request_id
>>>>>>>>>>>>>>>>> column; otherwise, insert null.
>>>>>>>>>>>>>>>>> - Add two new nullable columns, trace_id and
>>>>>> span_id,
>>>>>>>> and
>>>>>>>>>>>> populate
>>>>>>>>>>>>>>>>> them if OTel is enabled.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> From our discussions, I think it's important
>> to not
>>>>>>>> conflate
>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>> tracing fields with Envoy tracing fields,
>> which is
>>>> why
>>>>>> I
>>>>>>>>>> suggest we
>>>>>>>>>>>>>>>>> use separate fields / columns for them.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Would the above plan work for everyone?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw02wPvb0qxRzYAKEP0h8l9T&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3AIjY12CwTp7ItIvzTOIIN
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
>>>>>>>>>>>>>>>>> <ad...@snowflake.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks to Alex for starting this thread -
>> because
>>>> of
>>>>>>>> this, I’m
>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> coming to the realization that OTel Trace and
>> Span
>>>> IDs
>>>>>>> are
>>>>>>>>>> coming
>>>>>>>>>>>> built-in
>>>>>>>>>>>>>>>>> with Quarkus and my previous work to generate a
>>>>>>>>>>>> Request/Correlation ID is
>>>>>>>>>>>>>>>>> likely not needed as a result. My original
>>>> motivation
>>>>>> for
>>>>>>>>>>>> generation of a
>>>>>>>>>>>>>>>>> Request/Correlation ID was to ensure that any
>>>> client
>>>>>> can
>>>>>>>>>> uniquely
>>>>>>>>>>>> identify
>>>>>>>>>>>>>>>>> a request made to Polaris, which would be
>>>> especially
>>>>>>>> useful for
>>>>>>>>>>>> debugging
>>>>>>>>>>>>>>>>> failing requests or identifying call patterns.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we
>>>> should
>>>>>>>> remove
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Request/Correlation ID generation and always
>> use
>>>> the
>>>>>> OTel
>>>>>>>>>>>> trace/span IDs
>>>>>>>>>>>>>>>>> (which come for free with Quarkus) instead for
>> the
>>>>>>>> Correlation
>>>>>>>>>> ID
>>>>>>>>>>>> unless a
>>>>>>>>>>>>>>>>> valid header is present, which would take over
>> as
>>>> the
>>>>>>>>>> Correlation
>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>> instead.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To answer Dmitri’s question re Polaris
>> Events: The
>>>>>>>> intended
>>>>>>>>>> use
>>>>>>>>>>>> case is
>>>>>>>>>>>>>>>>> to provide some sort of correlation between
>> events
>>>> that
>>>>>>>> have
>>>>>>>>>>>> occurred as
>>>>>>>>>>>>>>>>> part of the same request. For example, if a
>> user
>>>> makes
>>>>>> an
>>>>>>>>>>>> CommitTransaction
>>>>>>>>>>>>>>>>> request, it would be helpful to see all
>> UpdateTable
>>>>>> calls
>>>>>>>> that
>>>>>>>>>>>> were made as
>>>>>>>>>>>>>>>>> part of that one user request.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>> Adnan Hemani
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
>>>> Bourlatchkov <
>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Logging x-request-id headers makes sense.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Just to confirm: if / when we restore that,
>>>> Polaris
>>>>>>> will
>>>>>>>> NOT
>>>>>>>>>>>> generate
>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> IDs in case the header is not present in the
>>>> request,
>>>>>>>>>> correct?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I believe x-request-id can co-exist with
>> OTel.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What about adding request IDs to events
>> [1][2]?
>>>>>> What's
>>>>>>>> the
>>>>>>>>>>>> intended use
>>>>>>>>>>>>>>>>>>> case for that? Could you share some context
>> here
>>>> too?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Side note: I proposed [2877] flagging event
>>>>>> persistence
>>>>>>>> as
>>>>>>>>>>>> "beta" in
>>>>>>>>>>>>>>>>>>> 1.2.0... This discussion adds another point
>>>> towards
>>>>>>>> that, I
>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%252523L97%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Pioys4ROm8vYM_mdx4ygH&source=gmail-imap&ust=1762450034000000&usg=AOvVaw39NasMg4Tgyrfe5Bj_U4xY
>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%252523L66%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw12-7e3ahm2sLSkLSNqTecm%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1PGW6uFwtUN8F1dOlJwrpm&source=gmail-imap&ust=1762450034000000&usg=AOvVaw3y_SObfpEJH_K_8RD9lVTL
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [2877]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2877%252523discussion_r2456300613%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2_e3B3qU8kJxdMmQx-q6bZ
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
>> Collado <
>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hey Dmitri
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The generating a request id is new code
>> that was
>>>>>> added
>>>>>>>>>> after the
>>>>>>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>> x-request-id support. You can see the state
>>>> from ~1
>>>>>>> year
>>>>>>>>>> ago, we
>>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>>>>> request_id as the header we used for the
>> MDC -
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%252523L415-L416%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw34DksOw7DSHJu8PfQQ5bT6&source=gmail-imap&ust=1762450034000000&usg=AOvVaw1g09FwhrTquJs9utV9CAX9
>>>>>>>>>>>>>>>>>>>> . At some point, it was changed to be
>>>> configurable,
>>>>>>>> then the
>>>>>>>>>>>>>>>>>>>> ContextResolverFilter filter was
>>>>>> refactored/eliminated
>>>>>>>> and
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but
>>>> lost
>>>>>> some
>>>>>>>> of
>>>>>>>>>> its
>>>>>>>>>>>> original
>>>>>>>>>>>>>>>>>>>> functionality.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> My personal opinion is that restoring
>> support
>>>> for
>>>>>> the
>>>>>>>>>>>> x-request-id
>>>>>>>>>>>>>>>>> header
>>>>>>>>>>>>>>>>>>>> is something that we should do, but if the
>>>> header
>>>>>>> isn't
>>>>>>>>>> present,
>>>>>>>>>>>>>>>>> falling
>>>>>>>>>>>>>>>>>>>> back on simply using OTel trace ids is good
>>>> enough
>>>>>>>> (better,
>>>>>>>>>>>> even) than
>>>>>>>>>>>>>>>>>>>> generating another random request id.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks for the info!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes
>>>> sense to
>>>>>>> me.
>>>>>>>>>>>> However, I
>>>>>>>>>>>>>>>>>>>> wonder:
>>>>>>>>>>>>>>>>>>>>> why would Polaris need to generate a new
>>>> request ID
>>>>>>>> inside
>>>>>>>>>> its
>>>>>>>>>>>>>>>>> code?..
>>>>>>>>>>>>>>>>>>>>> and return it in response headers?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> How important is it to propagate this ID to
>>>> Polaris
>>>>>>>> Events?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'm just trying to understand the full
>> context
>>>> :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
>>>> Collado <
>>>>>>>>>>>>>>>>> collado.mike@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I think the original intention for this
>>>> requestId
>>>>>>>> field
>>>>>>>>>> was to
>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>> request propagation from load balancers,
>> like
>>>>>> Envoy
>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw3LNqtilZ2OoJfE7yEwraZa&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2hTMXu2DU30Lg_5UBzsDJw
>>>>>>>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask
>> me
>>>> why
>>>>>> the
>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was
>>>> originally
>>>>>> a
>>>>>>>> custom
>>>>>>>>>>>> thing,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> we changed to integrate with existing
>>>> conventions.
>>>>>>>>>>>> Unfortunately,
>>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>> through the code, I think that the actual
>>>>>> functional
>>>>>>>>>> plumbing
>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>> lost
>>>>>>>>>>>>>>>>>>>>>> in the course of multiple refactors
>> around the
>>>>>> call
>>>>>>>>>> context
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> resolver. I
>>>>>>>>>>>>>>>>>>>>>> don't see references to that property or
>> the
>>>>>>>> underlying
>>>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Support for the unofficial x-request-id
>> header
>>>>>> feels
>>>>>>>> like
>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> should definitely keep, especially when
>>>> Polaris is
>>>>>>> one
>>>>>>>>>>>> service in a
>>>>>>>>>>>>>>>>>>>> mesh
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> services that maybe don't have OTel
>>>> integration.
>>>>>>> I'm a
>>>>>>>>>> fan of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>> standard, but it's not entirely
>> ubiquitous and
>>>>>> there
>>>>>>>> are
>>>>>>>>>> many
>>>>>>>>>>>>>>>>>>>> middleware
>>>>>>>>>>>>>>>>>>>>>> layers that know how to forward on the
>>>>>> x-request-id
>>>>>>>>>> header.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
>> Stupp <
>>>>>>>>>> snazy@snazy.de>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability
>> with
>>>> the
>>>>>>>> existing
>>>>>>>>>>>> de-facto
>>>>>>>>>>>>>>>>>>>>>>> standard OTel and make it easy for users
>> to
>>>>>>> integrate
>>>>>>>>>> into
>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>> observability platforms.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
>>>>>>> Bourlatchkov <
>>>>>>>>>>>>>>>>>>>> dimas@apache.org>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Alex and All,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I certainly support the idea of
>> following
>>>> OTel
>>>>>>>>>> standards for
>>>>>>>>>>>>>>>>>>>>> achieving
>>>>>>>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests
>> and/or
>>>>>> events.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> As to what form the correlation data
>> should
>>>>>> take,
>>>>>>> I
>>>>>>>>>> believe
>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>> conceptually what the OTel "context"
>>>> represents.
>>>>>>>> So, I
>>>>>>>>>>>> believe it
>>>>>>>>>>>>>>>>>>>>> makes
>>>>>>>>>>>>>>>>>>>>>>>> sense for Polaris to support standard
>>>> context
>>>>>>>>>> propagators
>>>>>>>>>>>> at the
>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request has OTel context
>>>>>>>> information,
>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>> returning
>>>>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>> other "correlation" data in the
>> response is
>>>>>>>> redundant, I
>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If the incoming request does not have
>> OTel
>>>>>> context
>>>>>>>> info,
>>>>>>>>>>>> what is
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific
>>>>>>>> "correlation
>>>>>>>>>> ID"?
>>>>>>>>>>>> How is
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> envisioned to be used?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If the intention is to correlate a
>> Polaris
>>>>>>> response
>>>>>>>>>>>> (operation)
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>> events
>>>>>>>>>>>>>>>>>>>>>>>> that might have resulted from its
>>>> execution, I
>>>>>>>> believe a
>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> robust
>>>>>>>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel
>>>> trace
>>>>>> info
>>>>>>>>>>>> (starting a new
>>>>>>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>> necessary) into event data. Then,
>> Polaris
>>>> can
>>>>>> also
>>>>>>>>>> return
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>> context
>>>>>>>>>>>>>>>>>>>>>>>> in the API response (top span). It's a
>> bit
>>>>>> awkward
>>>>>>>> from
>>>>>>>>>> the
>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>> perspective, but might be an option for
>>>>>> supporting
>>>>>>>>>> custom
>>>>>>>>>>>>>>>>>>>>> correlators.
>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The
>>>> header
>>>>>>> name
>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace
>> Context.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Custom correlation code will be able to
>>>> extract
>>>>>>> the
>>>>>>>>>> trace
>>>>>>>>>>>> ID from
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> response and from events and find
>> related
>>>> data.
>>>>>>>>>> Granted, it
>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> bit more effort for the custom code to
>>>> decode
>>>>>>> trace
>>>>>>>> IDs
>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>> OTel
>>>>>>>>>>>>>>>>>>>>>>>> context, but the format is standard and
>> not
>>>>>>>> complex. The
>>>>>>>>>>>> benefit
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily
>>>> integrate
>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> OTel-compatible
>>>>>>>>>>>>>>>>>>>>>>>> observability platforms regardless of
>>>> whether
>>>>>> any
>>>>>>>>>> particular
>>>>>>>>>>>>>>>>>>>>> deployment
>>>>>>>>>>>>>>>>>>>>>>>> uses custom correlators or not.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Dmitri.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
>> Alexandre
>>>> Dutra
>>>>>> <
>>>>>>>>>>>>>>>>> adutra@apache.org
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of
>> "request
>>>> ID",
>>>>>>> but
>>>>>>>> its
>>>>>>>>>>>> purpose is
>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an
>>>>>>>> observability
>>>>>>>>>>>> feature to
>>>>>>>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR
>> aims
>>>> to
>>>>>>>> rename it
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> "correlation
>>>>>>>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry
>>>>>> standards
>>>>>>>> [1].
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> However, this PR has brought to light
>>>> overlaps
>>>>>>> with
>>>>>>>>>> core
>>>>>>>>>>>>>>>>>>>> telemetry
>>>>>>>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is
>>>> enabled
>>>>>> in
>>>>>>>>>> Polaris,
>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>>>>>>> request already has a trace ID and
>> span ID,
>>>>>>> making
>>>>>>>> a
>>>>>>>>>>>> separate
>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID redundant.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and
>> span
>>>> ID
>>>>>> in
>>>>>>>>>> Polaris
>>>>>>>>>>>> events,
>>>>>>>>>>>>>>>>>>>>>>>>> rather than the generated correlation
>> ID,
>>>> would
>>>>>>>>>>>> significantly
>>>>>>>>>>>>>>>>>>>>>> simplify
>>>>>>>>>>>>>>>>>>>>>>>>> correlation of events with other
>> traces.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, I propose the following
>> changes:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace
>> ID and
>>>>>> span
>>>>>>>> ID as
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>>>>>>> ID for the request, instead of
>> generating a
>>>>>>> random
>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>> ID.
>>>>>>>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
>>>>>> correlation
>>>>>>>> ID
>>>>>>>>>>>> header is
>>>>>>>>>>>>>>>>>>>>>> present
>>>>>>>>>>>>>>>>>>>>>>>>> in the request, use it.
>>>>>>>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions
>> is
>>>> met,
>>>>>>>> generate
>>>>>>>>>> a
>>>>>>>>>>>> random
>>>>>>>>>>>>>>>>>>>>>>>>> correlation ID.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best
>>>> approach
>>>>>>> when a
>>>>>>>>>>>> correlation
>>>>>>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>>>>>>>>>>> header is present in the request.
>> However,
>>>> I
>>>>>>>> believe it
>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is
>>>> enabled, as
>>>>>>>> OTel
>>>>>>>>>>>> offers a
>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> robust solution for client-to-server
>> trace
>>>>>>>> propagation,
>>>>>>>>>>>> e.g. W3C
>>>>>>>>>>>>>>>>>>>>>> Trace
>>>>>>>>>>>>>>>>>>>>>>>>> Context propagation [2].
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Please share your thoughts!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> [1]:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2757%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Oe-25vtt4gLVSMFMtSVNg&source=gmail-imap&ust=1762450034000000&usg=AOvVaw2ohYKOQ8XU5-GBlDo6zYGC
>>>>>>>>>>>>>>>>>>>>>>>>> [2]:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1TRdkzABc_7U-_KZ1MZ59v&source=gmail-imap&ust=1762450034000000&usg=AOvVaw0cUBLtXCkRoH5pL57naUhY
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Alex,
SGTM (both not using request ID _response_ headers and deferring type info
in correlation ID values).
Cheers,
Dmitri.
On Thu, Oct 30, 2025 at 1:04 PM Alexandre Dutra <ad...@apache.org> wrote:
> Hi Dmitri,
>
> > I believe we still need to clarify / confirm whether returning any
> request / correlation ID in _response_ headers is required.
>
> Indeed, that specific item was still pending. FYI I'm going to
> *remove* the echoing of the request ID in the response headers, unless
> someone objects.
>
> > it could be useful to embed some type data into the value to help with
> interpretation
>
> I'm not sure we need this at this moment. I'd suggest that we leave
> this for further improvements in order to not over-specify the
> requirements. Given that the events feature is beta, this shouldn't be
> a problem.
>
> Thanks,
> Alex
>
>
> On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> >
> > Thanks for the nice summary, Alex!
> >
> > To All:
> >
> > I'm not sure an INDEX on correlation_id is that useful... Polaris itself
> > does not use that index, right? Do we expect end users to query events
> > directly from the database? This will work only with RDBMS, but probably
> > not with the proposed NoSQL persistence.
> >
> > I believe we still need to clarify / confirm whether returning any
> request
> > / correlation ID in _response_ headers is required.
> >
> > From my POV, I do not see a use case for it, but I wonder what other
> people
> > think / need.
> >
> > As for persisting correlation ID type, I agree that we do not need a
> > separate column for that. Still, it could be useful to embed some type
> data
> > into the value to help with interpretation, specifically for telemetry
> use
> > cases (both x-request-id and OTel). For example, we could
> > persist "traceparent:00-12345-...". Then, the reader would be able to
> > identify this as information following the W3C spec [1], parse it and
> link
> > to other traces (if desired). That processing is optional and at the
> > discretion of the event data reader, but making the value no-so-opaque
> > upfront can enable a lot of interesting options for users without much
> > extra overhead in Polaris. WDYT?
> >
> > [1] https://www.w3.org/TR/2021/REC-trace-context-1-20211123/
> >
> > Thanks,
> > Dmitri.
> >
> > On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> > > Hi all,
> > >
> > > > we would only persist one or the other, not both.
> > >
> > > My apologies, I confess that this wasn't my understanding of the
> desired
> > > design.
> > >
> > > Let's try to gather the requirements again:
> > >
> > > 1. Only one correlation ID is enough
> > > 2. The correlation ID is an opaque string
> > > 3. The main use case is to find events with matching correlation IDs
> > > 4. The only query pattern is by exact match
> > >
> > > Regarding *where* to store the correlation ID, and expanding on what
> > > Dmitri said: the requirement to find matching records by correlation
> > > ID clearly suggests a separate column, and with an INDEX on it.
> > > Storing it in additional_properties will be inefficient.
> > >
> > > If we all agree on the above, here is what I propose:
> > >
> > > 1. Rename the existing column request_id to correlation_id;
> > > 2. Populate from OTel context if available; the exact syntax is
> > > implementation-specific;
> > > 3. Otherwise, populate from X-Request-Id header if available;
> > > 4. Otherwise, don't populate.
> > > 5. Create an INDEX on correlation_id.
> > >
> > > You'll notice that we don't store the ID type, but that doesn't seem
> > > necessary.
> > >
> > > What do you all think?
> > >
> > > Thanks,
> > > Alex
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <di...@apache.org>
> > > wrote:
> > > >
> > > > Hi Yufei,
> > > >
> > > > [...] tying a BeforeUpdateTable event to the corresponding
> > > AfterUpdateTable
> > > > event.
> > > >
> > > >
> > > > It looks like this calls for a strong and always present ID in
> > > events.This
> > > > ID is obviously distincts from OTel context information and should be
> > > > persisted separately, IMHO.
> > > >
> > > > I believe there's still an open question whether the request ID in
> the
> > > > events should be populated from the x-request-id in REST API headers.
> > > >
> > > > Yufei: Is that required in your use cases?
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> > > >
> > > >
> > > > On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com>
> wrote:
> > > >
> > > > > I think the first question is whether any downstream use cases
> need to
> > > > > interpret the semantics of a request ID or OTel context. As I
> > > mentioned,
> > > > > the original purpose of a request ID was event correlation, e.g.,
> > > tying a
> > > > > BeforeUpdateTable event to the corresponding AfterUpdateTable
> event. If
> > > > > that’s the only requirement, an opaque correlation ID is good
> enough.
> > > We
> > > > > could change the field name(request-id) if needed.
> > > > >
> > > > > If we do have use cases that depend on the meaning of the request
> ID or
> > > > > OTel context, we can store them in additional_properties (
> > > > >
> > > > >
> > >
> https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134
> > > > > ).
> > > > > This is more flexible because it allows us to:
> > > > > 1. Record a key that indicates the format (e.g., x-request-id vs.
> > > > > otel-context).
> > > > > 2. Preserve both when Polaris receives both headers.
> > > > > 3. Add future tracing fields without a schema change.
> > > > >
> > > > > Yufei
> > > > >
> > > > >
> > > > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <
> dimas@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > From my POV the use cases for x-request-id and OTel are slightly
> > > > > different.
> > > > > > OTel is an observability use case where cross-linking of various
> > > pieces
> > > > > of
> > > > > > data is desired but not required in all cases (data loss is
> > > acceptable).
> > > > > > The x-request-id use case appears to have some very specific
> custom
> > > > > > applications without any obvious relationship to OTel.
> > > > > >
> > > > > > Given the current state of this discussion, I think it makes
> sense to
> > > > > > handle and persist them separately.
> > > > > >
> > > > > > However, if we intend x-request-id to act as an observability aid
> > > > > > alternative to OTel, then it might be preferable to have a config
> > > flag
> > > > > > indicating which one of them should be added to persistent event
> > > data. In
> > > > > > that case we should reuse the same column for them (and possibly
> add
> > > a
> > > > > > "type" prefix, e.g. "x-request-id:ABCDEF" or
> > > "traceparent:00-12345...")
> > > > > >
> > > > > > WDYT?
> > > > > >
> > > > > > Thanks,
> > > > > > Dmitri.
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <
> adutra@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Hi Yufei,
> > > > > > >
> > > > > > > > I’d suggest persisting a single opaque correlation ID instead
> > > > > > >
> > > > > > > The problem I see with this is that readers cannot know the
> syntax
> > > of
> > > > > > > the opaque id, and will have to "probe" using some heuristics,
> > > e.g. if
> > > > > > > the id matches the OTel trace context syntax, asume it's OTel,
> > > > > > > otherwise... assume it's something else.
> > > > > > >
> > > > > > > That's not unfeasible, but I wonder if it isn't better to store
> > > the 2
> > > > > > > ids separately.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <flyrain000@gmail.com
> >
> > > wrote:
> > > > > > > >
> > > > > > > > Thanks Alex for raising this discussion. Thanks everyone for
> > > chiming
> > > > > > in.
> > > > > > > > The plan looks good to me overall.
> > > > > > > >
> > > > > > > > > Update PolarisEvent, expose request ID if available, expose
> > > OTel
> > > > > > > context
> > > > > > > > if available.
> > > > > > > >
> > > > > > > > https://github.com/apache/polaris/pull/2914 persists both
> > > request id
> > > > > > and
> > > > > > > > otel context.
> > > > > > > >
> > > > > > > > Do we actually need to persist both?
> > > > > > > > From the original discussion, the intent of a request ID was
> to
> > > > > > correlate
> > > > > > > > multiple events generated by the same request, for example,
> > > linking a
> > > > > > > > BeforeUpdateTable and an AfterUpdateTable event. In that
> sense,
> > > what
> > > > > we
> > > > > > > > really need is a single opaque identifier that ties related
> > > events
> > > > > > > > together, regardless of whether it comes from a request ID
> or an
> > > OTel
> > > > > > > > context.
> > > > > > > >
> > > > > > > > If we persist both fields (requestId and otelContext), it can
> > > lead to
> > > > > > > > consumer confusion, since downstream systems would need to
> decide
> > > > > which
> > > > > > > one
> > > > > > > > to rely on, potentially causing inconsistent event
> correlation.
> > > > > Unless
> > > > > > > > there are concrete use cases where event consumers need
> both, I’d
> > > > > > suggest
> > > > > > > > persisting a single opaque correlation ID instead. This
> keeps the
> > > > > > schema
> > > > > > > > simpler and ensures consistent event linking.
> > > > > > > >
> > > > > > > > Yufei
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
> > > adutra@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Here is the PR:
> https://github.com/apache/polaris/pull/2914
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> > > > > > jb@nanthrax.net>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi
> > > > > > > > > >
> > > > > > > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > > > > > > >
> > > > > > > > > > Thanks !
> > > > > > > > > > Regards
> > > > > > > > > > JB
> > > > > > > > > >
> > > > > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
> > > adutra@apache.org
> > > > > >
> > > > > > a
> > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for the good discussion so far. I think we have
> a
> > > broad
> > > > > > > > > consensus.
> > > > > > > > > > >
> > > > > > > > > > > Unless anybody disagrees I am going to move forward
> with
> > > the
> > > > > the
> > > > > > > > > > > following revised plan, incorporating the latest
> feedback:
> > > > > > > > > > >
> > > > > > > > > > > 1. Restore the original functionality for request IDs,
> > > change
> > > > > the
> > > > > > > > > > > default header name back to x-request-id
> > > > > > > > > > >
> > > > > > > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > > > >
> > > > > > > > > > > 3. Update PolarisEvent, expose request ID if available,
> > > expose
> > > > > > OTel
> > > > > > > > > > > context if available.
> > > > > > > > > > >
> > > > > > > > > > > 4. Update events table SQL schema: insert request ID if
> > > > > > available,
> > > > > > > > > > > insert OTel context if available.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Alex
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > > > > > > jb@nanthrax.net
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi
> > > > > > > > > > > >
> > > > > > > > > > > > I think it's better to use w3c_trace_context (better
> > > than two
> > > > > > > > > fields).
> > > > > > > > > > > >
> > > > > > > > > > > > Regarding x-request-id, I think it's for a different
> > > purpose,
> > > > > > > > > > > > specifically for persistence. Personally, I think we
> can
> > > > > > achieve
> > > > > > > the
> > > > > > > > > > > > same with span and events.
> > > > > > > > > > > > If we want to "distinguish" the two layers, it makes
> > > sense to
> > > > > > > have
> > > > > > > > > > > > x-request-id (not sure it's super helpful). Else, I
> > > think we
> > > > > > can
> > > > > > > > > > > > achieve the same with span/event.
> > > > > > > > > > > >
> > > > > > > > > > > > Just my $0.01
> > > > > > > > > > > >
> > > > > > > > > > > > Regards
> > > > > > > > > > > > JB
> > > > > > > > > > > >
> > > > > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > >
> > > > > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested
> > > changes as
> > > > > > > well.
> > > > > > > > > Great
> > > > > > > > > > > find, Dmitri!
> > > > > > > > > > > > >
> > > > > > > > > > > > > I’m still debating with myself on whether we need
> to
> > > store
> > > > > > the
> > > > > > > > > > > `x-request-id` field as part of the Events persistence.
> > > Can we
> > > > > > > think
> > > > > > > > > of a
> > > > > > > > > > > good use case where this is more helpful to the user
> than
> > > the
> > > > > > OTel
> > > > > > > > > > > Trace/Span IDs? I am making the assumption here that
> those
> > > are
> > > > > > > still
> > > > > > > > > being
> > > > > > > > > > > returned back to the client.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Adnan Hemani
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > > > > > > adutra@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Dmitri,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Yes, I think your suggestion to use just one
> field
> > > > > > > > > w3c_trace_context
> > > > > > > > > > > > > > makes more sense than two fields (span_id and
> > > trace_id).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > With that, I also think that we are slowly
> drifting
> > > into
> > > > > > > > > > > > > > implementation considerations; let's get
> consensus
> > > on the
> > > > > > > general
> > > > > > > > > > > > > > design first, and we can certainly fine-tune the
> > > actual
> > > > > > Java
> > > > > > > > > methods
> > > > > > > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri
> Bourlatchkov <
> > > > > > > > > > > dimas@apache.org> wrote:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Hi All,
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Re: (3) Can we abstract request/trace info into
> a
> > > > > separate
> > > > > > > > > object
> > > > > > > > > > > without
> > > > > > > > > > > > > >> exposing those accessors on the Event class
> > > directly?
> > > > > OTel
> > > > > > > > > defines
> > > > > > > > > > > > > >> trace/span concepts, but in request ID is a bit
> > > foreign
> > > > > to
> > > > > > > OTel.
> > > > > > > > > > > Having
> > > > > > > > > > > > > >> tracing / request ID isolated in java could help
> > > with
> > > > > > > > > maintaining
> > > > > > > > > > > it and
> > > > > > > > > > > > > >> potentially supporting other (custom) tracing
> > > methods.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Re: (4) I'd like to propose storing OTel
> correlation
> > > > > data
> > > > > > > in the
> > > > > > > > > > > form of a
> > > > > > > > > > > > > >> standard context propagation string, e.g. W3C
> > > > > > trace-context
> > > > > > > [1]
> > > > > > > > > > > (same value
> > > > > > > > > > > > > >> as its HTTP header), so the column could be
> called
> > > > > > > > > > > w3c_trace_context or
> > > > > > > > > > > > > >> simply trace_context.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Open question: do we need to write a separate,
> > > > > individual
> > > > > > > trace
> > > > > > > > > ID
> > > > > > > > > > > field in
> > > > > > > > > > > > > >> SQL? I suppose it is not very useful since
> > > correlating
> > > > > it
> > > > > > to
> > > > > > > > > other
> > > > > > > > > > > trace
> > > > > > > > > > > > > >> data already requires understanding OTel context
> > > > > > propagation
> > > > > > > > > and a
> > > > > > > > > > > query
> > > > > > > > > > > > > >> against trace_context can still be made using
> > > > > > > string-matching
> > > > > > > > > > > clauses. We
> > > > > > > > > > > > > >> could probably (additionally) store it in the
> > > request_id
> > > > > > > column
> > > > > > > > > if
> > > > > > > > > > > the
> > > > > > > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> As for span ID, I do not really see a use case
> for
> > > > > > > persisting it
> > > > > > > > > > > > > >> individually. It is very specific to OTel trace
> data
> > > > > > > > > construction.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Actually, using the W3C trace context [1]
> encoding
> > > > > > probably
> > > > > > > > > makes
> > > > > > > > > > > sense in
> > > > > > > > > > > > > >> the java event representation too. Interested
> > > callers
> > > > > can
> > > > > > > easily
> > > > > > > > > > > decode
> > > > > > > > > > > > > >> this information since the format is
> well-defined.
> > > As a
> > > > > > side
> > > > > > > > > > > benefit, this
> > > > > > > > > > > > > >> opens opportunities for downstream event
> consumers
> > > to
> > > > > > > connect
> > > > > > > > > > > (propagate
> > > > > > > > > > > > > >> context) to traces that produced events based
> on the
> > > > > event
> > > > > > > data
> > > > > > > > > > > itself,
> > > > > > > > > > > > > >> without relying on the intermediate frameworks.
> > > This may
> > > > > > be
> > > > > > > > > > > desirable since
> > > > > > > > > > > > > >> the current Polaris event persistence impl.
> writes
> > > > > events
> > > > > > in
> > > > > > > > > > > batches, so
> > > > > > > > > > > > > >> the association to individual requests that
> produced
> > > > > those
> > > > > > > > > events
> > > > > > > > > > > is lost.
> > > > > > > > > > > > > >> Whether to perform this kind of context
> propagation
> > > will
> > > > > > be
> > > > > > > at
> > > > > > > > > the
> > > > > > > > > > > > > >> discretion of the event consumer, of course
> > > (outside of
> > > > > > > Polaris
> > > > > > > > > > > code).
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> [1]
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> WDYT?
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Thanks,
> > > > > > > > > > > > > >> Dmitri.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra
> <
> > > > > > > > > adutra@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>> Hi all,
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> Thank you for chiming in; the context around
> > > request
> > > > > IDs
> > > > > > > is now
> > > > > > > > > > > clear.
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> Trying to summarize this thread into actionable
> > > items,
> > > > > > > here's
> > > > > > > > > what
> > > > > > > > > > > I
> > > > > > > > > > > > > >>> propose:
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> 1. Restore the original functionality for
> request
> > > IDs.
> > > > > > > > > > > > > >>> - Change the default header name back to
> > > > > x-request-id
> > > > > > > > > (despite
> > > > > > > > > > > the
> > > > > > > > > > > > > >>> x- prefix being deprecated), but keep it
> > > configurable
> > > > > as
> > > > > > > today.
> > > > > > > > > > > > > >>> 2. Remove RequestIdGenerator and related
> > > functionality.
> > > > > > > > > > > > > >>> - Do not generate a request ID if the REST
> > > client
> > > > > > > doesn't
> > > > > > > > > > > provide one.
> > > > > > > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > > > > > > >>> - Expose new requestId(), traceId(),
> spanId()
> > > > > methods,
> > > > > > > all
> > > > > > > > > > > nullable.
> > > > > > > > > > > > > >>> - This would align with the emerging
> consensus
> > > > > around
> > > > > > > > > including
> > > > > > > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > > > > > > >>> - Insert the client-provided request ID
> into the
> > > > > > > request_id
> > > > > > > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > > > > > > >>> - Add two new nullable columns, trace_id and
> > > > > span_id,
> > > > > > > and
> > > > > > > > > > > populate
> > > > > > > > > > > > > >>> them if OTel is enabled.
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> From our discussions, I think it's important
> to not
> > > > > > > conflate
> > > > > > > > > OTel
> > > > > > > > > > > > > >>> tracing fields with Envoy tracing fields,
> which is
> > > why
> > > > > I
> > > > > > > > > suggest we
> > > > > > > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> Thanks,
> > > > > > > > > > > > > >>> Alex
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> [1]:
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> Hi all,
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> Thanks to Alex for starting this thread -
> because
> > > of
> > > > > > > this, I’m
> > > > > > > > > > > just
> > > > > > > > > > > > > >>> coming to the realization that OTel Trace and
> Span
> > > IDs
> > > > > > are
> > > > > > > > > coming
> > > > > > > > > > > built-in
> > > > > > > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > > > > > > Request/Correlation ID is
> > > > > > > > > > > > > >>> likely not needed as a result. My original
> > > motivation
> > > > > for
> > > > > > > > > > > generation of a
> > > > > > > > > > > > > >>> Request/Correlation ID was to ensure that any
> > > client
> > > > > can
> > > > > > > > > uniquely
> > > > > > > > > > > identify
> > > > > > > > > > > > > >>> a request made to Polaris, which would be
> > > especially
> > > > > > > useful for
> > > > > > > > > > > debugging
> > > > > > > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we
> > > should
> > > > > > > remove
> > > > > > > > > the
> > > > > > > > > > > > > >>> Request/Correlation ID generation and always
> use
> > > the
> > > > > OTel
> > > > > > > > > > > trace/span IDs
> > > > > > > > > > > > > >>> (which come for free with Quarkus) instead for
> the
> > > > > > > Correlation
> > > > > > > > > ID
> > > > > > > > > > > unless a
> > > > > > > > > > > > > >>> valid header is present, which would take over
> as
> > > the
> > > > > > > > > Correlation
> > > > > > > > > > > ID
> > > > > > > > > > > > > >>> instead.
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> —
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> To answer Dmitri’s question re Polaris
> Events: The
> > > > > > > intended
> > > > > > > > > use
> > > > > > > > > > > case is
> > > > > > > > > > > > > >>> to provide some sort of correlation between
> events
> > > that
> > > > > > > have
> > > > > > > > > > > occurred as
> > > > > > > > > > > > > >>> part of the same request. For example, if a
> user
> > > makes
> > > > > an
> > > > > > > > > > > CommitTransaction
> > > > > > > > > > > > > >>> request, it would be helpful to see all
> UpdateTable
> > > > > calls
> > > > > > > that
> > > > > > > > > > > were made as
> > > > > > > > > > > > > >>> part of that one user request.
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> Best,
> > > > > > > > > > > > > >>>> Adnan Hemani
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
> > > Bourlatchkov <
> > > > > > > > > > > dimas@apache.org>
> > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> Hi Michael,
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> Just to confirm: if / when we restore that,
> > > Polaris
> > > > > > will
> > > > > > > NOT
> > > > > > > > > > > generate
> > > > > > > > > > > > > >>> new
> > > > > > > > > > > > > >>>>> IDs in case the header is not present in the
> > > request,
> > > > > > > > > correct?
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> I believe x-request-id can co-exist with
> OTel.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> What about adding request IDs to events
> [1][2]?
> > > > > What's
> > > > > > > the
> > > > > > > > > > > intended use
> > > > > > > > > > > > > >>>>> case for that? Could you share some context
> here
> > > too?
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event
> > > > > persistence
> > > > > > > as
> > > > > > > > > > > "beta" in
> > > > > > > > > > > > > >>>>> 1.2.0... This discussion adds another point
> > > towards
> > > > > > > that, I
> > > > > > > > > > > think.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> [1]
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > > > > > > >>>>> [2]
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> [2877]
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> Thanks,
> > > > > > > > > > > > > >>>>> Dmitri.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael
> Collado <
> > > > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > > > >>>>> wrote:
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>> The generating a request id is new code
> that was
> > > > > added
> > > > > > > > > after the
> > > > > > > > > > > > > >>> original
> > > > > > > > > > > > > >>>>>> x-request-id support. You can see the state
> > > from ~1
> > > > > > year
> > > > > > > > > ago, we
> > > > > > > > > > > > > >>> hard-coded
> > > > > > > > > > > > > >>>>>> request_id as the header we used for the
> MDC -
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > > > > > > >>>>>> . At some point, it was changed to be
> > > configurable,
> > > > > > > then the
> > > > > > > > > > > > > >>>>>> ContextResolverFilter filter was
> > > > > refactored/eliminated
> > > > > > > and
> > > > > > > > > the
> > > > > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but
> > > lost
> > > > > some
> > > > > > > of
> > > > > > > > > its
> > > > > > > > > > > original
> > > > > > > > > > > > > >>>>>> functionality.
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>> My personal opinion is that restoring
> support
> > > for
> > > > > the
> > > > > > > > > > > x-request-id
> > > > > > > > > > > > > >>> header
> > > > > > > > > > > > > >>>>>> is something that we should do, but if the
> > > header
> > > > > > isn't
> > > > > > > > > present,
> > > > > > > > > > > > > >>> falling
> > > > > > > > > > > > > >>>>>> back on simply using OTel trace ids is good
> > > enough
> > > > > > > (better,
> > > > > > > > > > > even) than
> > > > > > > > > > > > > >>>>>> generating another random request id.
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>> Mike
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> > > > > Bourlatchkov <
> > > > > > > > > > > > > >>> dimas@apache.org>
> > > > > > > > > > > > > >>>>>> wrote:
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes
> > > sense to
> > > > > > me.
> > > > > > > > > > > However, I
> > > > > > > > > > > > > >>>>>> wonder:
> > > > > > > > > > > > > >>>>>>> why would Polaris need to generate a new
> > > request ID
> > > > > > > inside
> > > > > > > > > its
> > > > > > > > > > > > > >>> code?..
> > > > > > > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> How important is it to propagate this ID to
> > > Polaris
> > > > > > > Events?
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> I'm just trying to understand the full
> context
> > > :)
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> Thanks,
> > > > > > > > > > > > > >>>>>>> Dmitri.
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
> > > Collado <
> > > > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > > > >>>>>>> wrote:
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>>> I think the original intention for this
> > > requestId
> > > > > > > field
> > > > > > > > > was to
> > > > > > > > > > > > > >>> support
> > > > > > > > > > > > > >>>>>>>> request propagation from load balancers,
> like
> > > > > Envoy
> > > > > > (
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask
> me
> > > why
> > > > > the
> > > > > > > > > default
> > > > > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was
> > > originally
> > > > > a
> > > > > > > custom
> > > > > > > > > > > thing,
> > > > > > > > > > > > > >>> but
> > > > > > > > > > > > > >>>>>>> then
> > > > > > > > > > > > > >>>>>>>> we changed to integrate with existing
> > > conventions.
> > > > > > > > > > > Unfortunately,
> > > > > > > > > > > > > >>>>>> looking
> > > > > > > > > > > > > >>>>>>>> through the code, I think that the actual
> > > > > functional
> > > > > > > > > plumbing
> > > > > > > > > > > has
> > > > > > > > > > > > > >>> been
> > > > > > > > > > > > > >>>>>>> lost
> > > > > > > > > > > > > >>>>>>>> in the course of multiple refactors
> around the
> > > > > call
> > > > > > > > > context
> > > > > > > > > > > and
> > > > > > > > > > > > > >>>>>>> resolver. I
> > > > > > > > > > > > > >>>>>>>> don't see references to that property or
> the
> > > > > > > underlying
> > > > > > > > > > > header.
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id
> header
> > > > > feels
> > > > > > > like
> > > > > > > > > > > something
> > > > > > > > > > > > > >>> we
> > > > > > > > > > > > > >>>>>>>> should definitely keep, especially when
> > > Polaris is
> > > > > > one
> > > > > > > > > > > service in a
> > > > > > > > > > > > > >>>>>> mesh
> > > > > > > > > > > > > >>>>>>> of
> > > > > > > > > > > > > >>>>>>>> services that maybe don't have OTel
> > > integration.
> > > > > > I'm a
> > > > > > > > > fan of
> > > > > > > > > > > the
> > > > > > > > > > > > > >>> OTel
> > > > > > > > > > > > > >>>>>>>> standard, but it's not entirely
> ubiquitous and
> > > > > there
> > > > > > > are
> > > > > > > > > many
> > > > > > > > > > > > > >>>>>> middleware
> > > > > > > > > > > > > >>>>>>>> layers that know how to forward on the
> > > > > x-request-id
> > > > > > > > > header.
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>> Mike
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert
> Stupp <
> > > > > > > > > snazy@snazy.de>
> > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability
> with
> > > the
> > > > > > > existing
> > > > > > > > > > > de-facto
> > > > > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users
> to
> > > > > > integrate
> > > > > > > > > into
> > > > > > > > > > > their
> > > > > > > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> > > > > > Bourlatchkov <
> > > > > > > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> I certainly support the idea of
> following
> > > OTel
> > > > > > > > > standards for
> > > > > > > > > > > > > >>>>>>> achieving
> > > > > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests
> and/or
> > > > > events.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> As to what form the correlation data
> should
> > > > > take,
> > > > > > I
> > > > > > > > > believe
> > > > > > > > > > > it is
> > > > > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context"
> > > represents.
> > > > > > > So, I
> > > > > > > > > > > believe it
> > > > > > > > > > > > > >>>>>>> makes
> > > > > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard
> > > context
> > > > > > > > > propagators
> > > > > > > > > > > at the
> > > > > > > > > > > > > >>>>>> API
> > > > > > > > > > > > > >>>>>>>>> layer.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > > > > > > information,
> > > > > > > > > then
> > > > > > > > > > > > > >>>>>> returning
> > > > > > > > > > > > > >>>>>>>> any
> > > > > > > > > > > > > >>>>>>>>>> other "correlation" data in the
> response is
> > > > > > > redundant, I
> > > > > > > > > > > think.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> If the incoming request does not have
> OTel
> > > > > context
> > > > > > > info,
> > > > > > > > > > > what is
> > > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > > > > > > "correlation
> > > > > > > > > ID"?
> > > > > > > > > > > How is
> > > > > > > > > > > > > >>>>>> it
> > > > > > > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> If the intention is to correlate a
> Polaris
> > > > > > response
> > > > > > > > > > > (operation)
> > > > > > > > > > > > > >>>>>> with
> > > > > > > > > > > > > >>>>>>>>> events
> > > > > > > > > > > > > >>>>>>>>>> that might have resulted from its
> > > execution, I
> > > > > > > believe a
> > > > > > > > > > > more
> > > > > > > > > > > > > >>>>>> robust
> > > > > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel
> > > trace
> > > > > info
> > > > > > > > > > > (starting a new
> > > > > > > > > > > > > >>>>>>>> trace
> > > > > > > > > > > > > >>>>>>>>> if
> > > > > > > > > > > > > >>>>>>>>>> necessary) into event data. Then,
> Polaris
> > > can
> > > > > also
> > > > > > > > > return
> > > > > > > > > > > the
> > > > > > > > > > > > > >>> trace
> > > > > > > > > > > > > >>>>>>>>> context
> > > > > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a
> bit
> > > > > awkward
> > > > > > > from
> > > > > > > > > the
> > > > > > > > > > > OTel
> > > > > > > > > > > > > >>>>>>>>>> perspective, but might be an option for
> > > > > supporting
> > > > > > > > > custom
> > > > > > > > > > > > > >>>>>>> correlators.
> > > > > > > > > > > > > >>>>>>>> It
> > > > > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The
> > > header
> > > > > > name
> > > > > > > > > could be
> > > > > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace
> Context.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to
> > > extract
> > > > > > the
> > > > > > > > > trace
> > > > > > > > > > > ID from
> > > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > > >>>>>>>>>> response and from events and find
> related
> > > data.
> > > > > > > > > Granted, it
> > > > > > > > > > > will
> > > > > > > > > > > > > >>>>>>>> require
> > > > > > > > > > > > > >>>>>>>>> a
> > > > > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to
> > > decode
> > > > > > trace
> > > > > > > IDs
> > > > > > > > > > > from the
> > > > > > > > > > > > > >>>>>> OTel
> > > > > > > > > > > > > >>>>>>>>>> context, but the format is standard and
> not
> > > > > > > complex. The
> > > > > > > > > > > benefit
> > > > > > > > > > > > > >>>>>> for
> > > > > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily
> > > integrate
> > > > > > > with
> > > > > > > > > > > > > >>>>>> OTel-compatible
> > > > > > > > > > > > > >>>>>>>>>> observability platforms regardless of
> > > whether
> > > > > any
> > > > > > > > > particular
> > > > > > > > > > > > > >>>>>>> deployment
> > > > > > > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM
> Alexandre
> > > Dutra
> > > > > <
> > > > > > > > > > > > > >>> adutra@apache.org
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of
> "request
> > > ID",
> > > > > > but
> > > > > > > its
> > > > > > > > > > > purpose is
> > > > > > > > > > > > > >>>>>>> not
> > > > > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > > > > > > observability
> > > > > > > > > > > feature to
> > > > > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR
> aims
> > > to
> > > > > > > rename it
> > > > > > > > > to
> > > > > > > > > > > > > >>>>>>>> "correlation
> > > > > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry
> > > > > standards
> > > > > > > [1].
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light
> > > overlaps
> > > > > > with
> > > > > > > > > core
> > > > > > > > > > > > > >>>>>> telemetry
> > > > > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is
> > > enabled
> > > > > in
> > > > > > > > > Polaris,
> > > > > > > > > > > each
> > > > > > > > > > > > > >>>>>>>>>>> request already has a trace ID and
> span ID,
> > > > > > making
> > > > > > > a
> > > > > > > > > > > separate
> > > > > > > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and
> span
> > > ID
> > > > > in
> > > > > > > > > Polaris
> > > > > > > > > > > events,
> > > > > > > > > > > > > >>>>>>>>>>> rather than the generated correlation
> ID,
> > > would
> > > > > > > > > > > significantly
> > > > > > > > > > > > > >>>>>>>> simplify
> > > > > > > > > > > > > >>>>>>>>>>> correlation of events with other
> traces.
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following
> changes:
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace
> ID and
> > > > > span
> > > > > > > ID as
> > > > > > > > > the
> > > > > > > > > > > > > >>>>>>>> correlation
> > > > > > > > > > > > > >>>>>>>>>>> ID for the request, instead of
> generating a
> > > > > > random
> > > > > > > > > > > correlation
> > > > > > > > > > > > > >>>>>> ID.
> > > > > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> > > > > correlation
> > > > > > > ID
> > > > > > > > > > > header is
> > > > > > > > > > > > > >>>>>>>> present
> > > > > > > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions
> is
> > > met,
> > > > > > > generate
> > > > > > > > > a
> > > > > > > > > > > random
> > > > > > > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best
> > > approach
> > > > > > when a
> > > > > > > > > > > correlation
> > > > > > > > > > > > > >>>>>> ID
> > > > > > > > > > > > > >>>>>>>>>>> header is present in the request.
> However,
> > > I
> > > > > > > believe it
> > > > > > > > > > > would be
> > > > > > > > > > > > > >>>>>>> more
> > > > > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is
> > > enabled, as
> > > > > > > OTel
> > > > > > > > > > > offers a
> > > > > > > > > > > > > >>>>>> more
> > > > > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server
> trace
> > > > > > > propagation,
> > > > > > > > > > > e.g. W3C
> > > > > > > > > > > > > >>>>>>>> Trace
> > > > > > > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi Dmitri,
> I believe we still need to clarify / confirm whether returning any request / correlation ID in _response_ headers is required.
Indeed, that specific item was still pending. FYI I'm going to
*remove* the echoing of the request ID in the response headers, unless
someone objects.
> it could be useful to embed some type data into the value to help with interpretation
I'm not sure we need this at this moment. I'd suggest that we leave
this for further improvements in order to not over-specify the
requirements. Given that the events feature is beta, this shouldn't be
a problem.
Thanks,
Alex
On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Thanks for the nice summary, Alex!
>
> To All:
>
> I'm not sure an INDEX on correlation_id is that useful... Polaris itself
> does not use that index, right? Do we expect end users to query events
> directly from the database? This will work only with RDBMS, but probably
> not with the proposed NoSQL persistence.
>
> I believe we still need to clarify / confirm whether returning any request
> / correlation ID in _response_ headers is required.
>
> From my POV, I do not see a use case for it, but I wonder what other people
> think / need.
>
> As for persisting correlation ID type, I agree that we do not need a
> separate column for that. Still, it could be useful to embed some type data
> into the value to help with interpretation, specifically for telemetry use
> cases (both x-request-id and OTel). For example, we could
> persist "traceparent:00-12345-...". Then, the reader would be able to
> identify this as information following the W3C spec [1], parse it and link
> to other traces (if desired). That processing is optional and at the
> discretion of the event data reader, but making the value no-so-opaque
> upfront can enable a lot of interesting options for users without much
> extra overhead in Polaris. WDYT?
>
> [1] https://www.w3.org/TR/2021/REC-trace-context-1-20211123/
>
> Thanks,
> Dmitri.
>
> On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hi all,
> >
> > > we would only persist one or the other, not both.
> >
> > My apologies, I confess that this wasn't my understanding of the desired
> > design.
> >
> > Let's try to gather the requirements again:
> >
> > 1. Only one correlation ID is enough
> > 2. The correlation ID is an opaque string
> > 3. The main use case is to find events with matching correlation IDs
> > 4. The only query pattern is by exact match
> >
> > Regarding *where* to store the correlation ID, and expanding on what
> > Dmitri said: the requirement to find matching records by correlation
> > ID clearly suggests a separate column, and with an INDEX on it.
> > Storing it in additional_properties will be inefficient.
> >
> > If we all agree on the above, here is what I propose:
> >
> > 1. Rename the existing column request_id to correlation_id;
> > 2. Populate from OTel context if available; the exact syntax is
> > implementation-specific;
> > 3. Otherwise, populate from X-Request-Id header if available;
> > 4. Otherwise, don't populate.
> > 5. Create an INDEX on correlation_id.
> >
> > You'll notice that we don't store the ID type, but that doesn't seem
> > necessary.
> >
> > What do you all think?
> >
> > Thanks,
> > Alex
> >
> >
> >
> >
> >
> >
> > On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> > >
> > > Hi Yufei,
> > >
> > > [...] tying a BeforeUpdateTable event to the corresponding
> > AfterUpdateTable
> > > event.
> > >
> > >
> > > It looks like this calls for a strong and always present ID in
> > events.This
> > > ID is obviously distincts from OTel context information and should be
> > > persisted separately, IMHO.
> > >
> > > I believe there's still an open question whether the request ID in the
> > > events should be populated from the x-request-id in REST API headers.
> > >
> > > Yufei: Is that required in your use cases?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> > >
> > >
> > > On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com> wrote:
> > >
> > > > I think the first question is whether any downstream use cases need to
> > > > interpret the semantics of a request ID or OTel context. As I
> > mentioned,
> > > > the original purpose of a request ID was event correlation, e.g.,
> > tying a
> > > > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
> > > > that’s the only requirement, an opaque correlation ID is good enough.
> > We
> > > > could change the field name(request-id) if needed.
> > > >
> > > > If we do have use cases that depend on the meaning of the request ID or
> > > > OTel context, we can store them in additional_properties (
> > > >
> > > >
> > https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134
> > > > ).
> > > > This is more flexible because it allows us to:
> > > > 1. Record a key that indicates the format (e.g., x-request-id vs.
> > > > otel-context).
> > > > 2. Preserve both when Polaris receives both headers.
> > > > 3. Add future tracing fields without a schema change.
> > > >
> > > > Yufei
> > > >
> > > >
> > > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > From my POV the use cases for x-request-id and OTel are slightly
> > > > different.
> > > > > OTel is an observability use case where cross-linking of various
> > pieces
> > > > of
> > > > > data is desired but not required in all cases (data loss is
> > acceptable).
> > > > > The x-request-id use case appears to have some very specific custom
> > > > > applications without any obvious relationship to OTel.
> > > > >
> > > > > Given the current state of this discussion, I think it makes sense to
> > > > > handle and persist them separately.
> > > > >
> > > > > However, if we intend x-request-id to act as an observability aid
> > > > > alternative to OTel, then it might be preferable to have a config
> > flag
> > > > > indicating which one of them should be added to persistent event
> > data. In
> > > > > that case we should reuse the same column for them (and possibly add
> > a
> > > > > "type" prefix, e.g. "x-request-id:ABCDEF" or
> > "traceparent:00-12345...")
> > > > >
> > > > > WDYT?
> > > > >
> > > > > Thanks,
> > > > > Dmitri.
> > > > >
> > > > >
> > > > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org>
> > > > wrote:
> > > > >
> > > > > > Hi Yufei,
> > > > > >
> > > > > > > I’d suggest persisting a single opaque correlation ID instead
> > > > > >
> > > > > > The problem I see with this is that readers cannot know the syntax
> > of
> > > > > > the opaque id, and will have to "probe" using some heuristics,
> > e.g. if
> > > > > > the id matches the OTel trace context syntax, asume it's OTel,
> > > > > > otherwise... assume it's something else.
> > > > > >
> > > > > > That's not unfeasible, but I wonder if it isn't better to store
> > the 2
> > > > > > ids separately.
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > Thanks Alex for raising this discussion. Thanks everyone for
> > chiming
> > > > > in.
> > > > > > > The plan looks good to me overall.
> > > > > > >
> > > > > > > > Update PolarisEvent, expose request ID if available, expose
> > OTel
> > > > > > context
> > > > > > > if available.
> > > > > > >
> > > > > > > https://github.com/apache/polaris/pull/2914 persists both
> > request id
> > > > > and
> > > > > > > otel context.
> > > > > > >
> > > > > > > Do we actually need to persist both?
> > > > > > > From the original discussion, the intent of a request ID was to
> > > > > correlate
> > > > > > > multiple events generated by the same request, for example,
> > linking a
> > > > > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense,
> > what
> > > > we
> > > > > > > really need is a single opaque identifier that ties related
> > events
> > > > > > > together, regardless of whether it comes from a request ID or an
> > OTel
> > > > > > > context.
> > > > > > >
> > > > > > > If we persist both fields (requestId and otelContext), it can
> > lead to
> > > > > > > consumer confusion, since downstream systems would need to decide
> > > > which
> > > > > > one
> > > > > > > to rely on, potentially causing inconsistent event correlation.
> > > > Unless
> > > > > > > there are concrete use cases where event consumers need both, I’d
> > > > > suggest
> > > > > > > persisting a single opaque correlation ID instead. This keeps the
> > > > > schema
> > > > > > > simpler and ensures consistent event linking.
> > > > > > >
> > > > > > > Yufei
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
> > adutra@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> > > > > jb@nanthrax.net>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi
> > > > > > > > >
> > > > > > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > > > > > >
> > > > > > > > > Thanks !
> > > > > > > > > Regards
> > > > > > > > > JB
> > > > > > > > >
> > > > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
> > adutra@apache.org
> > > > >
> > > > > a
> > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > Thanks for the good discussion so far. I think we have a
> > broad
> > > > > > > > consensus.
> > > > > > > > > >
> > > > > > > > > > Unless anybody disagrees I am going to move forward with
> > the
> > > > the
> > > > > > > > > > following revised plan, incorporating the latest feedback:
> > > > > > > > > >
> > > > > > > > > > 1. Restore the original functionality for request IDs,
> > change
> > > > the
> > > > > > > > > > default header name back to x-request-id
> > > > > > > > > >
> > > > > > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > > >
> > > > > > > > > > 3. Update PolarisEvent, expose request ID if available,
> > expose
> > > > > OTel
> > > > > > > > > > context if available.
> > > > > > > > > >
> > > > > > > > > > 4. Update events table SQL schema: insert request ID if
> > > > > available,
> > > > > > > > > > insert OTel context if available.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Alex
> > > > > > > > > >
> > > > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > > > > > jb@nanthrax.net
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi
> > > > > > > > > > >
> > > > > > > > > > > I think it's better to use w3c_trace_context (better
> > than two
> > > > > > > > fields).
> > > > > > > > > > >
> > > > > > > > > > > Regarding x-request-id, I think it's for a different
> > purpose,
> > > > > > > > > > > specifically for persistence. Personally, I think we can
> > > > > achieve
> > > > > > the
> > > > > > > > > > > same with span and events.
> > > > > > > > > > > If we want to "distinguish" the two layers, it makes
> > sense to
> > > > > > have
> > > > > > > > > > > x-request-id (not sure it's super helpful). Else, I
> > think we
> > > > > can
> > > > > > > > > > > achieve the same with span/event.
> > > > > > > > > > >
> > > > > > > > > > > Just my $0.01
> > > > > > > > > > >
> > > > > > > > > > > Regards
> > > > > > > > > > > JB
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi all,
> > > > > > > > > > > >
> > > > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested
> > changes as
> > > > > > well.
> > > > > > > > Great
> > > > > > > > > > find, Dmitri!
> > > > > > > > > > > >
> > > > > > > > > > > > I’m still debating with myself on whether we need to
> > store
> > > > > the
> > > > > > > > > > `x-request-id` field as part of the Events persistence.
> > Can we
> > > > > > think
> > > > > > > > of a
> > > > > > > > > > good use case where this is more helpful to the user than
> > the
> > > > > OTel
> > > > > > > > > > Trace/Span IDs? I am making the assumption here that those
> > are
> > > > > > still
> > > > > > > > being
> > > > > > > > > > returned back to the client.
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Adnan Hemani
> > > > > > > > > > > >
> > > > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > > > > > adutra@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi Dmitri,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes, I think your suggestion to use just one field
> > > > > > > > w3c_trace_context
> > > > > > > > > > > > > makes more sense than two fields (span_id and
> > trace_id).
> > > > > > > > > > > > >
> > > > > > > > > > > > > With that, I also think that we are slowly drifting
> > into
> > > > > > > > > > > > > implementation considerations; let's get consensus
> > on the
> > > > > > general
> > > > > > > > > > > > > design first, and we can certainly fine-tune the
> > actual
> > > > > Java
> > > > > > > > methods
> > > > > > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > Alex
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > > > > > > dimas@apache.org> wrote:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Hi All,
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Re: (3) Can we abstract request/trace info into a
> > > > separate
> > > > > > > > object
> > > > > > > > > > without
> > > > > > > > > > > > >> exposing those accessors on the Event class
> > directly?
> > > > OTel
> > > > > > > > defines
> > > > > > > > > > > > >> trace/span concepts, but in request ID is a bit
> > foreign
> > > > to
> > > > > > OTel.
> > > > > > > > > > Having
> > > > > > > > > > > > >> tracing / request ID isolated in java could help
> > with
> > > > > > > > maintaining
> > > > > > > > > > it and
> > > > > > > > > > > > >> potentially supporting other (custom) tracing
> > methods.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation
> > > > data
> > > > > > in the
> > > > > > > > > > form of a
> > > > > > > > > > > > >> standard context propagation string, e.g. W3C
> > > > > trace-context
> > > > > > [1]
> > > > > > > > > > (same value
> > > > > > > > > > > > >> as its HTTP header), so the column could be called
> > > > > > > > > > w3c_trace_context or
> > > > > > > > > > > > >> simply trace_context.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Open question: do we need to write a separate,
> > > > individual
> > > > > > trace
> > > > > > > > ID
> > > > > > > > > > field in
> > > > > > > > > > > > >> SQL? I suppose it is not very useful since
> > correlating
> > > > it
> > > > > to
> > > > > > > > other
> > > > > > > > > > trace
> > > > > > > > > > > > >> data already requires understanding OTel context
> > > > > propagation
> > > > > > > > and a
> > > > > > > > > > query
> > > > > > > > > > > > >> against trace_context can still be made using
> > > > > > string-matching
> > > > > > > > > > clauses. We
> > > > > > > > > > > > >> could probably (additionally) store it in the
> > request_id
> > > > > > column
> > > > > > > > if
> > > > > > > > > > the
> > > > > > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> As for span ID, I do not really see a use case for
> > > > > > persisting it
> > > > > > > > > > > > >> individually. It is very specific to OTel trace data
> > > > > > > > construction.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Actually, using the W3C trace context [1] encoding
> > > > > probably
> > > > > > > > makes
> > > > > > > > > > sense in
> > > > > > > > > > > > >> the java event representation too. Interested
> > callers
> > > > can
> > > > > > easily
> > > > > > > > > > decode
> > > > > > > > > > > > >> this information since the format is well-defined.
> > As a
> > > > > side
> > > > > > > > > > benefit, this
> > > > > > > > > > > > >> opens opportunities for downstream event consumers
> > to
> > > > > > connect
> > > > > > > > > > (propagate
> > > > > > > > > > > > >> context) to traces that produced events based on the
> > > > event
> > > > > > data
> > > > > > > > > > itself,
> > > > > > > > > > > > >> without relying on the intermediate frameworks.
> > This may
> > > > > be
> > > > > > > > > > desirable since
> > > > > > > > > > > > >> the current Polaris event persistence impl. writes
> > > > events
> > > > > in
> > > > > > > > > > batches, so
> > > > > > > > > > > > >> the association to individual requests that produced
> > > > those
> > > > > > > > events
> > > > > > > > > > is lost.
> > > > > > > > > > > > >> Whether to perform this kind of context propagation
> > will
> > > > > be
> > > > > > at
> > > > > > > > the
> > > > > > > > > > > > >> discretion of the event consumer, of course
> > (outside of
> > > > > > Polaris
> > > > > > > > > > code).
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> [1]
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> WDYT?
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Thanks,
> > > > > > > > > > > > >> Dmitri.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > > > > > > adutra@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>> Hi all,
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Thank you for chiming in; the context around
> > request
> > > > IDs
> > > > > > is now
> > > > > > > > > > clear.
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Trying to summarize this thread into actionable
> > items,
> > > > > > here's
> > > > > > > > what
> > > > > > > > > > I
> > > > > > > > > > > > >>> propose:
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> 1. Restore the original functionality for request
> > IDs.
> > > > > > > > > > > > >>> - Change the default header name back to
> > > > x-request-id
> > > > > > > > (despite
> > > > > > > > > > the
> > > > > > > > > > > > >>> x- prefix being deprecated), but keep it
> > configurable
> > > > as
> > > > > > today.
> > > > > > > > > > > > >>> 2. Remove RequestIdGenerator and related
> > functionality.
> > > > > > > > > > > > >>> - Do not generate a request ID if the REST
> > client
> > > > > > doesn't
> > > > > > > > > > provide one.
> > > > > > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > > > > > >>> - Expose new requestId(), traceId(), spanId()
> > > > methods,
> > > > > > all
> > > > > > > > > > nullable.
> > > > > > > > > > > > >>> - This would align with the emerging consensus
> > > > around
> > > > > > > > including
> > > > > > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > > > > > >>> - Insert the client-provided request ID into the
> > > > > > request_id
> > > > > > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > > > > > >>> - Add two new nullable columns, trace_id and
> > > > span_id,
> > > > > > and
> > > > > > > > > > populate
> > > > > > > > > > > > >>> them if OTel is enabled.
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> From our discussions, I think it's important to not
> > > > > > conflate
> > > > > > > > OTel
> > > > > > > > > > > > >>> tracing fields with Envoy tracing fields, which is
> > why
> > > > I
> > > > > > > > suggest we
> > > > > > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Thanks,
> > > > > > > > > > > > >>> Alex
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> [1]:
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> Hi all,
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> Thanks to Alex for starting this thread - because
> > of
> > > > > > this, I’m
> > > > > > > > > > just
> > > > > > > > > > > > >>> coming to the realization that OTel Trace and Span
> > IDs
> > > > > are
> > > > > > > > coming
> > > > > > > > > > built-in
> > > > > > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > > > > > Request/Correlation ID is
> > > > > > > > > > > > >>> likely not needed as a result. My original
> > motivation
> > > > for
> > > > > > > > > > generation of a
> > > > > > > > > > > > >>> Request/Correlation ID was to ensure that any
> > client
> > > > can
> > > > > > > > uniquely
> > > > > > > > > > identify
> > > > > > > > > > > > >>> a request made to Polaris, which would be
> > especially
> > > > > > useful for
> > > > > > > > > > debugging
> > > > > > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we
> > should
> > > > > > remove
> > > > > > > > the
> > > > > > > > > > > > >>> Request/Correlation ID generation and always use
> > the
> > > > OTel
> > > > > > > > > > trace/span IDs
> > > > > > > > > > > > >>> (which come for free with Quarkus) instead for the
> > > > > > Correlation
> > > > > > > > ID
> > > > > > > > > > unless a
> > > > > > > > > > > > >>> valid header is present, which would take over as
> > the
> > > > > > > > Correlation
> > > > > > > > > > ID
> > > > > > > > > > > > >>> instead.
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> —
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> > > > > > intended
> > > > > > > > use
> > > > > > > > > > case is
> > > > > > > > > > > > >>> to provide some sort of correlation between events
> > that
> > > > > > have
> > > > > > > > > > occurred as
> > > > > > > > > > > > >>> part of the same request. For example, if a user
> > makes
> > > > an
> > > > > > > > > > CommitTransaction
> > > > > > > > > > > > >>> request, it would be helpful to see all UpdateTable
> > > > calls
> > > > > > that
> > > > > > > > > > were made as
> > > > > > > > > > > > >>> part of that one user request.
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> Best,
> > > > > > > > > > > > >>>> Adnan Hemani
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
> > Bourlatchkov <
> > > > > > > > > > dimas@apache.org>
> > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> Hi Michael,
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> Just to confirm: if / when we restore that,
> > Polaris
> > > > > will
> > > > > > NOT
> > > > > > > > > > generate
> > > > > > > > > > > > >>> new
> > > > > > > > > > > > >>>>> IDs in case the header is not present in the
> > request,
> > > > > > > > correct?
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> What about adding request IDs to events [1][2]?
> > > > What's
> > > > > > the
> > > > > > > > > > intended use
> > > > > > > > > > > > >>>>> case for that? Could you share some context here
> > too?
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event
> > > > persistence
> > > > > > as
> > > > > > > > > > "beta" in
> > > > > > > > > > > > >>>>> 1.2.0... This discussion adds another point
> > towards
> > > > > > that, I
> > > > > > > > > > think.
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> [1]
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > > > > > >>>>> [2]
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> [2877]
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> Thanks,
> > > > > > > > > > > > >>>>> Dmitri.
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > > >>>>> wrote:
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>> The generating a request id is new code that was
> > > > added
> > > > > > > > after the
> > > > > > > > > > > > >>> original
> > > > > > > > > > > > >>>>>> x-request-id support. You can see the state
> > from ~1
> > > > > year
> > > > > > > > ago, we
> > > > > > > > > > > > >>> hard-coded
> > > > > > > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > > > > > >>>>>> . At some point, it was changed to be
> > configurable,
> > > > > > then the
> > > > > > > > > > > > >>>>>> ContextResolverFilter filter was
> > > > refactored/eliminated
> > > > > > and
> > > > > > > > the
> > > > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but
> > lost
> > > > some
> > > > > > of
> > > > > > > > its
> > > > > > > > > > original
> > > > > > > > > > > > >>>>>> functionality.
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>> My personal opinion is that restoring support
> > for
> > > > the
> > > > > > > > > > x-request-id
> > > > > > > > > > > > >>> header
> > > > > > > > > > > > >>>>>> is something that we should do, but if the
> > header
> > > > > isn't
> > > > > > > > present,
> > > > > > > > > > > > >>> falling
> > > > > > > > > > > > >>>>>> back on simply using OTel trace ids is good
> > enough
> > > > > > (better,
> > > > > > > > > > even) than
> > > > > > > > > > > > >>>>>> generating another random request id.
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>> Mike
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> > > > Bourlatchkov <
> > > > > > > > > > > > >>> dimas@apache.org>
> > > > > > > > > > > > >>>>>> wrote:
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes
> > sense to
> > > > > me.
> > > > > > > > > > However, I
> > > > > > > > > > > > >>>>>> wonder:
> > > > > > > > > > > > >>>>>>> why would Polaris need to generate a new
> > request ID
> > > > > > inside
> > > > > > > > its
> > > > > > > > > > > > >>> code?..
> > > > > > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> How important is it to propagate this ID to
> > Polaris
> > > > > > Events?
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> I'm just trying to understand the full context
> > :)
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> Thanks,
> > > > > > > > > > > > >>>>>>> Dmitri.
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
> > Collado <
> > > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > > >>>>>>> wrote:
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>>> I think the original intention for this
> > requestId
> > > > > > field
> > > > > > > > was to
> > > > > > > > > > > > >>> support
> > > > > > > > > > > > >>>>>>>> request propagation from load balancers, like
> > > > Envoy
> > > > > (
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me
> > why
> > > > the
> > > > > > > > default
> > > > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was
> > originally
> > > > a
> > > > > > custom
> > > > > > > > > > thing,
> > > > > > > > > > > > >>> but
> > > > > > > > > > > > >>>>>>> then
> > > > > > > > > > > > >>>>>>>> we changed to integrate with existing
> > conventions.
> > > > > > > > > > Unfortunately,
> > > > > > > > > > > > >>>>>> looking
> > > > > > > > > > > > >>>>>>>> through the code, I think that the actual
> > > > functional
> > > > > > > > plumbing
> > > > > > > > > > has
> > > > > > > > > > > > >>> been
> > > > > > > > > > > > >>>>>>> lost
> > > > > > > > > > > > >>>>>>>> in the course of multiple refactors around the
> > > > call
> > > > > > > > context
> > > > > > > > > > and
> > > > > > > > > > > > >>>>>>> resolver. I
> > > > > > > > > > > > >>>>>>>> don't see references to that property or the
> > > > > > underlying
> > > > > > > > > > header.
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header
> > > > feels
> > > > > > like
> > > > > > > > > > something
> > > > > > > > > > > > >>> we
> > > > > > > > > > > > >>>>>>>> should definitely keep, especially when
> > Polaris is
> > > > > one
> > > > > > > > > > service in a
> > > > > > > > > > > > >>>>>> mesh
> > > > > > > > > > > > >>>>>>> of
> > > > > > > > > > > > >>>>>>>> services that maybe don't have OTel
> > integration.
> > > > > I'm a
> > > > > > > > fan of
> > > > > > > > > > the
> > > > > > > > > > > > >>> OTel
> > > > > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and
> > > > there
> > > > > > are
> > > > > > > > many
> > > > > > > > > > > > >>>>>> middleware
> > > > > > > > > > > > >>>>>>>> layers that know how to forward on the
> > > > x-request-id
> > > > > > > > header.
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>> Mike
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > > > > > > snazy@snazy.de>
> > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with
> > the
> > > > > > existing
> > > > > > > > > > de-facto
> > > > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to
> > > > > integrate
> > > > > > > > into
> > > > > > > > > > their
> > > > > > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> > > > > Bourlatchkov <
> > > > > > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> I certainly support the idea of following
> > OTel
> > > > > > > > standards for
> > > > > > > > > > > > >>>>>>> achieving
> > > > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or
> > > > events.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> As to what form the correlation data should
> > > > take,
> > > > > I
> > > > > > > > believe
> > > > > > > > > > it is
> > > > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context"
> > represents.
> > > > > > So, I
> > > > > > > > > > believe it
> > > > > > > > > > > > >>>>>>> makes
> > > > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard
> > context
> > > > > > > > propagators
> > > > > > > > > > at the
> > > > > > > > > > > > >>>>>> API
> > > > > > > > > > > > >>>>>>>>> layer.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > > > > > information,
> > > > > > > > then
> > > > > > > > > > > > >>>>>> returning
> > > > > > > > > > > > >>>>>>>> any
> > > > > > > > > > > > >>>>>>>>>> other "correlation" data in the response is
> > > > > > redundant, I
> > > > > > > > > > think.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel
> > > > context
> > > > > > info,
> > > > > > > > > > what is
> > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > > > > > "correlation
> > > > > > > > ID"?
> > > > > > > > > > How is
> > > > > > > > > > > > >>>>>> it
> > > > > > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris
> > > > > response
> > > > > > > > > > (operation)
> > > > > > > > > > > > >>>>>> with
> > > > > > > > > > > > >>>>>>>>> events
> > > > > > > > > > > > >>>>>>>>>> that might have resulted from its
> > execution, I
> > > > > > believe a
> > > > > > > > > > more
> > > > > > > > > > > > >>>>>> robust
> > > > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel
> > trace
> > > > info
> > > > > > > > > > (starting a new
> > > > > > > > > > > > >>>>>>>> trace
> > > > > > > > > > > > >>>>>>>>> if
> > > > > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris
> > can
> > > > also
> > > > > > > > return
> > > > > > > > > > the
> > > > > > > > > > > > >>> trace
> > > > > > > > > > > > >>>>>>>>> context
> > > > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit
> > > > awkward
> > > > > > from
> > > > > > > > the
> > > > > > > > > > OTel
> > > > > > > > > > > > >>>>>>>>>> perspective, but might be an option for
> > > > supporting
> > > > > > > > custom
> > > > > > > > > > > > >>>>>>> correlators.
> > > > > > > > > > > > >>>>>>>> It
> > > > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The
> > header
> > > > > name
> > > > > > > > could be
> > > > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to
> > extract
> > > > > the
> > > > > > > > trace
> > > > > > > > > > ID from
> > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > >>>>>>>>>> response and from events and find related
> > data.
> > > > > > > > Granted, it
> > > > > > > > > > will
> > > > > > > > > > > > >>>>>>>> require
> > > > > > > > > > > > >>>>>>>>> a
> > > > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to
> > decode
> > > > > trace
> > > > > > IDs
> > > > > > > > > > from the
> > > > > > > > > > > > >>>>>> OTel
> > > > > > > > > > > > >>>>>>>>>> context, but the format is standard and not
> > > > > > complex. The
> > > > > > > > > > benefit
> > > > > > > > > > > > >>>>>> for
> > > > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily
> > integrate
> > > > > > with
> > > > > > > > > > > > >>>>>> OTel-compatible
> > > > > > > > > > > > >>>>>>>>>> observability platforms regardless of
> > whether
> > > > any
> > > > > > > > particular
> > > > > > > > > > > > >>>>>>> deployment
> > > > > > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre
> > Dutra
> > > > <
> > > > > > > > > > > > >>> adutra@apache.org
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request
> > ID",
> > > > > but
> > > > > > its
> > > > > > > > > > purpose is
> > > > > > > > > > > > >>>>>>> not
> > > > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > > > > > observability
> > > > > > > > > > feature to
> > > > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims
> > to
> > > > > > rename it
> > > > > > > > to
> > > > > > > > > > > > >>>>>>>> "correlation
> > > > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry
> > > > standards
> > > > > > [1].
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light
> > overlaps
> > > > > with
> > > > > > > > core
> > > > > > > > > > > > >>>>>> telemetry
> > > > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is
> > enabled
> > > > in
> > > > > > > > Polaris,
> > > > > > > > > > each
> > > > > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID,
> > > > > making
> > > > > > a
> > > > > > > > > > separate
> > > > > > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span
> > ID
> > > > in
> > > > > > > > Polaris
> > > > > > > > > > events,
> > > > > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID,
> > would
> > > > > > > > > > significantly
> > > > > > > > > > > > >>>>>>>> simplify
> > > > > > > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and
> > > > span
> > > > > > ID as
> > > > > > > > the
> > > > > > > > > > > > >>>>>>>> correlation
> > > > > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a
> > > > > random
> > > > > > > > > > correlation
> > > > > > > > > > > > >>>>>> ID.
> > > > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> > > > correlation
> > > > > > ID
> > > > > > > > > > header is
> > > > > > > > > > > > >>>>>>>> present
> > > > > > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is
> > met,
> > > > > > generate
> > > > > > > > a
> > > > > > > > > > random
> > > > > > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best
> > approach
> > > > > when a
> > > > > > > > > > correlation
> > > > > > > > > > > > >>>>>> ID
> > > > > > > > > > > > >>>>>>>>>>> header is present in the request. However,
> > I
> > > > > > believe it
> > > > > > > > > > would be
> > > > > > > > > > > > >>>>>>> more
> > > > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is
> > enabled, as
> > > > > > OTel
> > > > > > > > > > offers a
> > > > > > > > > > > > >>>>>> more
> > > > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> > > > > > propagation,
> > > > > > > > > > e.g. W3C
> > > > > > > > > > > > >>>>>>>> Trace
> > > > > > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Thanks for the nice summary, Alex!
To All:
I'm not sure an INDEX on correlation_id is that useful... Polaris itself
does not use that index, right? Do we expect end users to query events
directly from the database? This will work only with RDBMS, but probably
not with the proposed NoSQL persistence.
I believe we still need to clarify / confirm whether returning any request
/ correlation ID in _response_ headers is required.
From my POV, I do not see a use case for it, but I wonder what other people
think / need.
As for persisting correlation ID type, I agree that we do not need a
separate column for that. Still, it could be useful to embed some type data
into the value to help with interpretation, specifically for telemetry use
cases (both x-request-id and OTel). For example, we could
persist "traceparent:00-12345-...". Then, the reader would be able to
identify this as information following the W3C spec [1], parse it and link
to other traces (if desired). That processing is optional and at the
discretion of the event data reader, but making the value no-so-opaque
upfront can enable a lot of interesting options for users without much
extra overhead in Polaris. WDYT?
[1] https://www.w3.org/TR/2021/REC-trace-context-1-20211123/
Thanks,
Dmitri.
On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <ad...@apache.org> wrote:
> Hi all,
>
> > we would only persist one or the other, not both.
>
> My apologies, I confess that this wasn't my understanding of the desired
> design.
>
> Let's try to gather the requirements again:
>
> 1. Only one correlation ID is enough
> 2. The correlation ID is an opaque string
> 3. The main use case is to find events with matching correlation IDs
> 4. The only query pattern is by exact match
>
> Regarding *where* to store the correlation ID, and expanding on what
> Dmitri said: the requirement to find matching records by correlation
> ID clearly suggests a separate column, and with an INDEX on it.
> Storing it in additional_properties will be inefficient.
>
> If we all agree on the above, here is what I propose:
>
> 1. Rename the existing column request_id to correlation_id;
> 2. Populate from OTel context if available; the exact syntax is
> implementation-specific;
> 3. Otherwise, populate from X-Request-Id header if available;
> 4. Otherwise, don't populate.
> 5. Create an INDEX on correlation_id.
>
> You'll notice that we don't store the ID type, but that doesn't seem
> necessary.
>
> What do you all think?
>
> Thanks,
> Alex
>
>
>
>
>
>
> On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> >
> > Hi Yufei,
> >
> > [...] tying a BeforeUpdateTable event to the corresponding
> AfterUpdateTable
> > event.
> >
> >
> > It looks like this calls for a strong and always present ID in
> events.This
> > ID is obviously distincts from OTel context information and should be
> > persisted separately, IMHO.
> >
> > I believe there's still an open question whether the request ID in the
> > events should be populated from the x-request-id in REST API headers.
> >
> > Yufei: Is that required in your use cases?
> >
> > Thanks,
> > Dmitri.
> >
> >
> >
> > On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com> wrote:
> >
> > > I think the first question is whether any downstream use cases need to
> > > interpret the semantics of a request ID or OTel context. As I
> mentioned,
> > > the original purpose of a request ID was event correlation, e.g.,
> tying a
> > > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
> > > that’s the only requirement, an opaque correlation ID is good enough.
> We
> > > could change the field name(request-id) if needed.
> > >
> > > If we do have use cases that depend on the meaning of the request ID or
> > > OTel context, we can store them in additional_properties (
> > >
> > >
> https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134
> > > ).
> > > This is more flexible because it allows us to:
> > > 1. Record a key that indicates the format (e.g., x-request-id vs.
> > > otel-context).
> > > 2. Preserve both when Polaris receives both headers.
> > > 3. Add future tracing fields without a schema change.
> > >
> > > Yufei
> > >
> > >
> > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > From my POV the use cases for x-request-id and OTel are slightly
> > > different.
> > > > OTel is an observability use case where cross-linking of various
> pieces
> > > of
> > > > data is desired but not required in all cases (data loss is
> acceptable).
> > > > The x-request-id use case appears to have some very specific custom
> > > > applications without any obvious relationship to OTel.
> > > >
> > > > Given the current state of this discussion, I think it makes sense to
> > > > handle and persist them separately.
> > > >
> > > > However, if we intend x-request-id to act as an observability aid
> > > > alternative to OTel, then it might be preferable to have a config
> flag
> > > > indicating which one of them should be added to persistent event
> data. In
> > > > that case we should reuse the same column for them (and possibly add
> a
> > > > "type" prefix, e.g. "x-request-id:ABCDEF" or
> "traceparent:00-12345...")
> > > >
> > > > WDYT?
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> > > >
> > > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org>
> > > wrote:
> > > >
> > > > > Hi Yufei,
> > > > >
> > > > > > I’d suggest persisting a single opaque correlation ID instead
> > > > >
> > > > > The problem I see with this is that readers cannot know the syntax
> of
> > > > > the opaque id, and will have to "probe" using some heuristics,
> e.g. if
> > > > > the id matches the OTel trace context syntax, asume it's OTel,
> > > > > otherwise... assume it's something else.
> > > > >
> > > > > That's not unfeasible, but I wonder if it isn't better to store
> the 2
> > > > > ids separately.
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com>
> wrote:
> > > > > >
> > > > > > Thanks Alex for raising this discussion. Thanks everyone for
> chiming
> > > > in.
> > > > > > The plan looks good to me overall.
> > > > > >
> > > > > > > Update PolarisEvent, expose request ID if available, expose
> OTel
> > > > > context
> > > > > > if available.
> > > > > >
> > > > > > https://github.com/apache/polaris/pull/2914 persists both
> request id
> > > > and
> > > > > > otel context.
> > > > > >
> > > > > > Do we actually need to persist both?
> > > > > > From the original discussion, the intent of a request ID was to
> > > > correlate
> > > > > > multiple events generated by the same request, for example,
> linking a
> > > > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense,
> what
> > > we
> > > > > > really need is a single opaque identifier that ties related
> events
> > > > > > together, regardless of whether it comes from a request ID or an
> OTel
> > > > > > context.
> > > > > >
> > > > > > If we persist both fields (requestId and otelContext), it can
> lead to
> > > > > > consumer confusion, since downstream systems would need to decide
> > > which
> > > > > one
> > > > > > to rely on, potentially causing inconsistent event correlation.
> > > Unless
> > > > > > there are concrete use cases where event consumers need both, I’d
> > > > suggest
> > > > > > persisting a single opaque correlation ID instead. This keeps the
> > > > schema
> > > > > > simpler and ensures consistent event linking.
> > > > > >
> > > > > > Yufei
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <
> adutra@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> > > > jb@nanthrax.net>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi
> > > > > > > >
> > > > > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > > > > >
> > > > > > > > Thanks !
> > > > > > > > Regards
> > > > > > > > JB
> > > > > > > >
> > > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <
> adutra@apache.org
> > > >
> > > > a
> > > > > > > écrit :
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Thanks for the good discussion so far. I think we have a
> broad
> > > > > > > consensus.
> > > > > > > > >
> > > > > > > > > Unless anybody disagrees I am going to move forward with
> the
> > > the
> > > > > > > > > following revised plan, incorporating the latest feedback:
> > > > > > > > >
> > > > > > > > > 1. Restore the original functionality for request IDs,
> change
> > > the
> > > > > > > > > default header name back to x-request-id
> > > > > > > > >
> > > > > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > >
> > > > > > > > > 3. Update PolarisEvent, expose request ID if available,
> expose
> > > > OTel
> > > > > > > > > context if available.
> > > > > > > > >
> > > > > > > > > 4. Update events table SQL schema: insert request ID if
> > > > available,
> > > > > > > > > insert OTel context if available.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > > > > jb@nanthrax.net
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi
> > > > > > > > > >
> > > > > > > > > > I think it's better to use w3c_trace_context (better
> than two
> > > > > > > fields).
> > > > > > > > > >
> > > > > > > > > > Regarding x-request-id, I think it's for a different
> purpose,
> > > > > > > > > > specifically for persistence. Personally, I think we can
> > > > achieve
> > > > > the
> > > > > > > > > > same with span and events.
> > > > > > > > > > If we want to "distinguish" the two layers, it makes
> sense to
> > > > > have
> > > > > > > > > > x-request-id (not sure it's super helpful). Else, I
> think we
> > > > can
> > > > > > > > > > achieve the same with span/event.
> > > > > > > > > >
> > > > > > > > > > Just my $0.01
> > > > > > > > > >
> > > > > > > > > > Regards
> > > > > > > > > > JB
> > > > > > > > > >
> > > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested
> changes as
> > > > > well.
> > > > > > > Great
> > > > > > > > > find, Dmitri!
> > > > > > > > > > >
> > > > > > > > > > > I’m still debating with myself on whether we need to
> store
> > > > the
> > > > > > > > > `x-request-id` field as part of the Events persistence.
> Can we
> > > > > think
> > > > > > > of a
> > > > > > > > > good use case where this is more helpful to the user than
> the
> > > > OTel
> > > > > > > > > Trace/Span IDs? I am making the assumption here that those
> are
> > > > > still
> > > > > > > being
> > > > > > > > > returned back to the client.
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Adnan Hemani
> > > > > > > > > > >
> > > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > > > > adutra@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Dmitri,
> > > > > > > > > > > >
> > > > > > > > > > > > Yes, I think your suggestion to use just one field
> > > > > > > w3c_trace_context
> > > > > > > > > > > > makes more sense than two fields (span_id and
> trace_id).
> > > > > > > > > > > >
> > > > > > > > > > > > With that, I also think that we are slowly drifting
> into
> > > > > > > > > > > > implementation considerations; let's get consensus
> on the
> > > > > general
> > > > > > > > > > > > design first, and we can certainly fine-tune the
> actual
> > > > Java
> > > > > > > methods
> > > > > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Alex
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > > > > > dimas@apache.org> wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >> Hi All,
> > > > > > > > > > > >>
> > > > > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > > > > >>
> > > > > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > > > > >>
> > > > > > > > > > > >> Re: (3) Can we abstract request/trace info into a
> > > separate
> > > > > > > object
> > > > > > > > > without
> > > > > > > > > > > >> exposing those accessors on the Event class
> directly?
> > > OTel
> > > > > > > defines
> > > > > > > > > > > >> trace/span concepts, but in request ID is a bit
> foreign
> > > to
> > > > > OTel.
> > > > > > > > > Having
> > > > > > > > > > > >> tracing / request ID isolated in java could help
> with
> > > > > > > maintaining
> > > > > > > > > it and
> > > > > > > > > > > >> potentially supporting other (custom) tracing
> methods.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation
> > > data
> > > > > in the
> > > > > > > > > form of a
> > > > > > > > > > > >> standard context propagation string, e.g. W3C
> > > > trace-context
> > > > > [1]
> > > > > > > > > (same value
> > > > > > > > > > > >> as its HTTP header), so the column could be called
> > > > > > > > > w3c_trace_context or
> > > > > > > > > > > >> simply trace_context.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Open question: do we need to write a separate,
> > > individual
> > > > > trace
> > > > > > > ID
> > > > > > > > > field in
> > > > > > > > > > > >> SQL? I suppose it is not very useful since
> correlating
> > > it
> > > > to
> > > > > > > other
> > > > > > > > > trace
> > > > > > > > > > > >> data already requires understanding OTel context
> > > > propagation
> > > > > > > and a
> > > > > > > > > query
> > > > > > > > > > > >> against trace_context can still be made using
> > > > > string-matching
> > > > > > > > > clauses. We
> > > > > > > > > > > >> could probably (additionally) store it in the
> request_id
> > > > > column
> > > > > > > if
> > > > > > > > > the
> > > > > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > > > > >>
> > > > > > > > > > > >> As for span ID, I do not really see a use case for
> > > > > persisting it
> > > > > > > > > > > >> individually. It is very specific to OTel trace data
> > > > > > > construction.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Actually, using the W3C trace context [1] encoding
> > > > probably
> > > > > > > makes
> > > > > > > > > sense in
> > > > > > > > > > > >> the java event representation too. Interested
> callers
> > > can
> > > > > easily
> > > > > > > > > decode
> > > > > > > > > > > >> this information since the format is well-defined.
> As a
> > > > side
> > > > > > > > > benefit, this
> > > > > > > > > > > >> opens opportunities for downstream event consumers
> to
> > > > > connect
> > > > > > > > > (propagate
> > > > > > > > > > > >> context) to traces that produced events based on the
> > > event
> > > > > data
> > > > > > > > > itself,
> > > > > > > > > > > >> without relying on the intermediate frameworks.
> This may
> > > > be
> > > > > > > > > desirable since
> > > > > > > > > > > >> the current Polaris event persistence impl. writes
> > > events
> > > > in
> > > > > > > > > batches, so
> > > > > > > > > > > >> the association to individual requests that produced
> > > those
> > > > > > > events
> > > > > > > > > is lost.
> > > > > > > > > > > >> Whether to perform this kind of context propagation
> will
> > > > be
> > > > > at
> > > > > > > the
> > > > > > > > > > > >> discretion of the event consumer, of course
> (outside of
> > > > > Polaris
> > > > > > > > > code).
> > > > > > > > > > > >>
> > > > > > > > > > > >> [1]
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > > > > >>
> > > > > > > > > > > >> WDYT?
> > > > > > > > > > > >>
> > > > > > > > > > > >> Thanks,
> > > > > > > > > > > >> Dmitri.
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > > > > > adutra@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> Hi all,
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Thank you for chiming in; the context around
> request
> > > IDs
> > > > > is now
> > > > > > > > > clear.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Trying to summarize this thread into actionable
> items,
> > > > > here's
> > > > > > > what
> > > > > > > > > I
> > > > > > > > > > > >>> propose:
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> 1. Restore the original functionality for request
> IDs.
> > > > > > > > > > > >>> - Change the default header name back to
> > > x-request-id
> > > > > > > (despite
> > > > > > > > > the
> > > > > > > > > > > >>> x- prefix being deprecated), but keep it
> configurable
> > > as
> > > > > today.
> > > > > > > > > > > >>> 2. Remove RequestIdGenerator and related
> functionality.
> > > > > > > > > > > >>> - Do not generate a request ID if the REST
> client
> > > > > doesn't
> > > > > > > > > provide one.
> > > > > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > > > > >>> - Expose new requestId(), traceId(), spanId()
> > > methods,
> > > > > all
> > > > > > > > > nullable.
> > > > > > > > > > > >>> - This would align with the emerging consensus
> > > around
> > > > > > > including
> > > > > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > > > > >>> - Insert the client-provided request ID into the
> > > > > request_id
> > > > > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > > > > >>> - Add two new nullable columns, trace_id and
> > > span_id,
> > > > > and
> > > > > > > > > populate
> > > > > > > > > > > >>> them if OTel is enabled.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> From our discussions, I think it's important to not
> > > > > conflate
> > > > > > > OTel
> > > > > > > > > > > >>> tracing fields with Envoy tracing fields, which is
> why
> > > I
> > > > > > > suggest we
> > > > > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Thanks,
> > > > > > > > > > > >>> Alex
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> [1]:
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> Hi all,
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> Thanks to Alex for starting this thread - because
> of
> > > > > this, I’m
> > > > > > > > > just
> > > > > > > > > > > >>> coming to the realization that OTel Trace and Span
> IDs
> > > > are
> > > > > > > coming
> > > > > > > > > built-in
> > > > > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > > > > Request/Correlation ID is
> > > > > > > > > > > >>> likely not needed as a result. My original
> motivation
> > > for
> > > > > > > > > generation of a
> > > > > > > > > > > >>> Request/Correlation ID was to ensure that any
> client
> > > can
> > > > > > > uniquely
> > > > > > > > > identify
> > > > > > > > > > > >>> a request made to Polaris, which would be
> especially
> > > > > useful for
> > > > > > > > > debugging
> > > > > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we
> should
> > > > > remove
> > > > > > > the
> > > > > > > > > > > >>> Request/Correlation ID generation and always use
> the
> > > OTel
> > > > > > > > > trace/span IDs
> > > > > > > > > > > >>> (which come for free with Quarkus) instead for the
> > > > > Correlation
> > > > > > > ID
> > > > > > > > > unless a
> > > > > > > > > > > >>> valid header is present, which would take over as
> the
> > > > > > > Correlation
> > > > > > > > > ID
> > > > > > > > > > > >>> instead.
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> —
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> > > > > intended
> > > > > > > use
> > > > > > > > > case is
> > > > > > > > > > > >>> to provide some sort of correlation between events
> that
> > > > > have
> > > > > > > > > occurred as
> > > > > > > > > > > >>> part of the same request. For example, if a user
> makes
> > > an
> > > > > > > > > CommitTransaction
> > > > > > > > > > > >>> request, it would be helpful to see all UpdateTable
> > > calls
> > > > > that
> > > > > > > > > were made as
> > > > > > > > > > > >>> part of that one user request.
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> Best,
> > > > > > > > > > > >>>> Adnan Hemani
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri
> Bourlatchkov <
> > > > > > > > > dimas@apache.org>
> > > > > > > > > > > >>> wrote:
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> Hi Michael,
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> Just to confirm: if / when we restore that,
> Polaris
> > > > will
> > > > > NOT
> > > > > > > > > generate
> > > > > > > > > > > >>> new
> > > > > > > > > > > >>>>> IDs in case the header is not present in the
> request,
> > > > > > > correct?
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> What about adding request IDs to events [1][2]?
> > > What's
> > > > > the
> > > > > > > > > intended use
> > > > > > > > > > > >>>>> case for that? Could you share some context here
> too?
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event
> > > persistence
> > > > > as
> > > > > > > > > "beta" in
> > > > > > > > > > > >>>>> 1.2.0... This discussion adds another point
> towards
> > > > > that, I
> > > > > > > > > think.
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> [1]
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > > > > >>>>> [2]
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> [2877]
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> Thanks,
> > > > > > > > > > > >>>>> Dmitri.
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > >>>>> wrote:
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>> The generating a request id is new code that was
> > > added
> > > > > > > after the
> > > > > > > > > > > >>> original
> > > > > > > > > > > >>>>>> x-request-id support. You can see the state
> from ~1
> > > > year
> > > > > > > ago, we
> > > > > > > > > > > >>> hard-coded
> > > > > > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > > > > >>>>>> . At some point, it was changed to be
> configurable,
> > > > > then the
> > > > > > > > > > > >>>>>> ContextResolverFilter filter was
> > > refactored/eliminated
> > > > > and
> > > > > > > the
> > > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but
> lost
> > > some
> > > > > of
> > > > > > > its
> > > > > > > > > original
> > > > > > > > > > > >>>>>> functionality.
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>> My personal opinion is that restoring support
> for
> > > the
> > > > > > > > > x-request-id
> > > > > > > > > > > >>> header
> > > > > > > > > > > >>>>>> is something that we should do, but if the
> header
> > > > isn't
> > > > > > > present,
> > > > > > > > > > > >>> falling
> > > > > > > > > > > >>>>>> back on simply using OTel trace ids is good
> enough
> > > > > (better,
> > > > > > > > > even) than
> > > > > > > > > > > >>>>>> generating another random request id.
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>> Mike
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> > > Bourlatchkov <
> > > > > > > > > > > >>> dimas@apache.org>
> > > > > > > > > > > >>>>>> wrote:
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes
> sense to
> > > > me.
> > > > > > > > > However, I
> > > > > > > > > > > >>>>>> wonder:
> > > > > > > > > > > >>>>>>> why would Polaris need to generate a new
> request ID
> > > > > inside
> > > > > > > its
> > > > > > > > > > > >>> code?..
> > > > > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> How important is it to propagate this ID to
> Polaris
> > > > > Events?
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> I'm just trying to understand the full context
> :)
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> Thanks,
> > > > > > > > > > > >>>>>>> Dmitri.
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael
> Collado <
> > > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > > >>>>>>> wrote:
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>>> I think the original intention for this
> requestId
> > > > > field
> > > > > > > was to
> > > > > > > > > > > >>> support
> > > > > > > > > > > >>>>>>>> request propagation from load balancers, like
> > > Envoy
> > > > (
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me
> why
> > > the
> > > > > > > default
> > > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was
> originally
> > > a
> > > > > custom
> > > > > > > > > thing,
> > > > > > > > > > > >>> but
> > > > > > > > > > > >>>>>>> then
> > > > > > > > > > > >>>>>>>> we changed to integrate with existing
> conventions.
> > > > > > > > > Unfortunately,
> > > > > > > > > > > >>>>>> looking
> > > > > > > > > > > >>>>>>>> through the code, I think that the actual
> > > functional
> > > > > > > plumbing
> > > > > > > > > has
> > > > > > > > > > > >>> been
> > > > > > > > > > > >>>>>>> lost
> > > > > > > > > > > >>>>>>>> in the course of multiple refactors around the
> > > call
> > > > > > > context
> > > > > > > > > and
> > > > > > > > > > > >>>>>>> resolver. I
> > > > > > > > > > > >>>>>>>> don't see references to that property or the
> > > > > underlying
> > > > > > > > > header.
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header
> > > feels
> > > > > like
> > > > > > > > > something
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>>>>>>> should definitely keep, especially when
> Polaris is
> > > > one
> > > > > > > > > service in a
> > > > > > > > > > > >>>>>> mesh
> > > > > > > > > > > >>>>>>> of
> > > > > > > > > > > >>>>>>>> services that maybe don't have OTel
> integration.
> > > > I'm a
> > > > > > > fan of
> > > > > > > > > the
> > > > > > > > > > > >>> OTel
> > > > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and
> > > there
> > > > > are
> > > > > > > many
> > > > > > > > > > > >>>>>> middleware
> > > > > > > > > > > >>>>>>>> layers that know how to forward on the
> > > x-request-id
> > > > > > > header.
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>> Mike
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > > > > > snazy@snazy.de>
> > > > > > > > > > > >>> wrote:
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with
> the
> > > > > existing
> > > > > > > > > de-facto
> > > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to
> > > > integrate
> > > > > > > into
> > > > > > > > > their
> > > > > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> > > > Bourlatchkov <
> > > > > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> I certainly support the idea of following
> OTel
> > > > > > > standards for
> > > > > > > > > > > >>>>>>> achieving
> > > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or
> > > events.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> As to what form the correlation data should
> > > take,
> > > > I
> > > > > > > believe
> > > > > > > > > it is
> > > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context"
> represents.
> > > > > So, I
> > > > > > > > > believe it
> > > > > > > > > > > >>>>>>> makes
> > > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard
> context
> > > > > > > propagators
> > > > > > > > > at the
> > > > > > > > > > > >>>>>> API
> > > > > > > > > > > >>>>>>>>> layer.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > > > > information,
> > > > > > > then
> > > > > > > > > > > >>>>>> returning
> > > > > > > > > > > >>>>>>>> any
> > > > > > > > > > > >>>>>>>>>> other "correlation" data in the response is
> > > > > redundant, I
> > > > > > > > > think.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel
> > > context
> > > > > info,
> > > > > > > > > what is
> > > > > > > > > > > >>>>>> the
> > > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > > > > "correlation
> > > > > > > ID"?
> > > > > > > > > How is
> > > > > > > > > > > >>>>>> it
> > > > > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris
> > > > response
> > > > > > > > > (operation)
> > > > > > > > > > > >>>>>> with
> > > > > > > > > > > >>>>>>>>> events
> > > > > > > > > > > >>>>>>>>>> that might have resulted from its
> execution, I
> > > > > believe a
> > > > > > > > > more
> > > > > > > > > > > >>>>>> robust
> > > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel
> trace
> > > info
> > > > > > > > > (starting a new
> > > > > > > > > > > >>>>>>>> trace
> > > > > > > > > > > >>>>>>>>> if
> > > > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris
> can
> > > also
> > > > > > > return
> > > > > > > > > the
> > > > > > > > > > > >>> trace
> > > > > > > > > > > >>>>>>>>> context
> > > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit
> > > awkward
> > > > > from
> > > > > > > the
> > > > > > > > > OTel
> > > > > > > > > > > >>>>>>>>>> perspective, but might be an option for
> > > supporting
> > > > > > > custom
> > > > > > > > > > > >>>>>>> correlators.
> > > > > > > > > > > >>>>>>>> It
> > > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The
> header
> > > > name
> > > > > > > could be
> > > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to
> extract
> > > > the
> > > > > > > trace
> > > > > > > > > ID from
> > > > > > > > > > > >>>>>> the
> > > > > > > > > > > >>>>>>>>>> response and from events and find related
> data.
> > > > > > > Granted, it
> > > > > > > > > will
> > > > > > > > > > > >>>>>>>> require
> > > > > > > > > > > >>>>>>>>> a
> > > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to
> decode
> > > > trace
> > > > > IDs
> > > > > > > > > from the
> > > > > > > > > > > >>>>>> OTel
> > > > > > > > > > > >>>>>>>>>> context, but the format is standard and not
> > > > > complex. The
> > > > > > > > > benefit
> > > > > > > > > > > >>>>>> for
> > > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily
> integrate
> > > > > with
> > > > > > > > > > > >>>>>> OTel-compatible
> > > > > > > > > > > >>>>>>>>>> observability platforms regardless of
> whether
> > > any
> > > > > > > particular
> > > > > > > > > > > >>>>>>> deployment
> > > > > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre
> Dutra
> > > <
> > > > > > > > > > > >>> adutra@apache.org
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request
> ID",
> > > > but
> > > > > its
> > > > > > > > > purpose is
> > > > > > > > > > > >>>>>>> not
> > > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > > > > observability
> > > > > > > > > feature to
> > > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims
> to
> > > > > rename it
> > > > > > > to
> > > > > > > > > > > >>>>>>>> "correlation
> > > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry
> > > standards
> > > > > [1].
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light
> overlaps
> > > > with
> > > > > > > core
> > > > > > > > > > > >>>>>> telemetry
> > > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is
> enabled
> > > in
> > > > > > > Polaris,
> > > > > > > > > each
> > > > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID,
> > > > making
> > > > > a
> > > > > > > > > separate
> > > > > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span
> ID
> > > in
> > > > > > > Polaris
> > > > > > > > > events,
> > > > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID,
> would
> > > > > > > > > significantly
> > > > > > > > > > > >>>>>>>> simplify
> > > > > > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and
> > > span
> > > > > ID as
> > > > > > > the
> > > > > > > > > > > >>>>>>>> correlation
> > > > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a
> > > > random
> > > > > > > > > correlation
> > > > > > > > > > > >>>>>> ID.
> > > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> > > correlation
> > > > > ID
> > > > > > > > > header is
> > > > > > > > > > > >>>>>>>> present
> > > > > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is
> met,
> > > > > generate
> > > > > > > a
> > > > > > > > > random
> > > > > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best
> approach
> > > > when a
> > > > > > > > > correlation
> > > > > > > > > > > >>>>>> ID
> > > > > > > > > > > >>>>>>>>>>> header is present in the request. However,
> I
> > > > > believe it
> > > > > > > > > would be
> > > > > > > > > > > >>>>>>> more
> > > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is
> enabled, as
> > > > > OTel
> > > > > > > > > offers a
> > > > > > > > > > > >>>>>> more
> > > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> > > > > propagation,
> > > > > > > > > e.g. W3C
> > > > > > > > > > > >>>>>>>> Trace
> > > > > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >>>>>>>
> > > > > > > > > > > >>>>>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,
> we would only persist one or the other, not both.
My apologies, I confess that this wasn't my understanding of the desired design.
Let's try to gather the requirements again:
1. Only one correlation ID is enough
2. The correlation ID is an opaque string
3. The main use case is to find events with matching correlation IDs
4. The only query pattern is by exact match
Regarding *where* to store the correlation ID, and expanding on what
Dmitri said: the requirement to find matching records by correlation
ID clearly suggests a separate column, and with an INDEX on it.
Storing it in additional_properties will be inefficient.
If we all agree on the above, here is what I propose:
1. Rename the existing column request_id to correlation_id;
2. Populate from OTel context if available; the exact syntax is
implementation-specific;
3. Otherwise, populate from X-Request-Id header if available;
4. Otherwise, don't populate.
5. Create an INDEX on correlation_id.
You'll notice that we don't store the ID type, but that doesn't seem necessary.
What do you all think?
Thanks,
Alex
On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Yufei,
>
> [...] tying a BeforeUpdateTable event to the corresponding AfterUpdateTable
> event.
>
>
> It looks like this calls for a strong and always present ID in events.This
> ID is obviously distincts from OTel context information and should be
> persisted separately, IMHO.
>
> I believe there's still an open question whether the request ID in the
> events should be populated from the x-request-id in REST API headers.
>
> Yufei: Is that required in your use cases?
>
> Thanks,
> Dmitri.
>
>
>
> On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com> wrote:
>
> > I think the first question is whether any downstream use cases need to
> > interpret the semantics of a request ID or OTel context. As I mentioned,
> > the original purpose of a request ID was event correlation, e.g., tying a
> > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
> > that’s the only requirement, an opaque correlation ID is good enough. We
> > could change the field name(request-id) if needed.
> >
> > If we do have use cases that depend on the meaning of the request ID or
> > OTel context, we can store them in additional_properties (
> >
> > https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134
> > ).
> > This is more flexible because it allows us to:
> > 1. Record a key that indicates the format (e.g., x-request-id vs.
> > otel-context).
> > 2. Preserve both when Polaris receives both headers.
> > 3. Add future tracing fields without a schema change.
> >
> > Yufei
> >
> >
> > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> >
> > > Hi All,
> > >
> > > From my POV the use cases for x-request-id and OTel are slightly
> > different.
> > > OTel is an observability use case where cross-linking of various pieces
> > of
> > > data is desired but not required in all cases (data loss is acceptable).
> > > The x-request-id use case appears to have some very specific custom
> > > applications without any obvious relationship to OTel.
> > >
> > > Given the current state of this discussion, I think it makes sense to
> > > handle and persist them separately.
> > >
> > > However, if we intend x-request-id to act as an observability aid
> > > alternative to OTel, then it might be preferable to have a config flag
> > > indicating which one of them should be added to persistent event data. In
> > > that case we should reuse the same column for them (and possibly add a
> > > "type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...")
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> > >
> > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > >
> > > > Hi Yufei,
> > > >
> > > > > I’d suggest persisting a single opaque correlation ID instead
> > > >
> > > > The problem I see with this is that readers cannot know the syntax of
> > > > the opaque id, and will have to "probe" using some heuristics, e.g. if
> > > > the id matches the OTel trace context syntax, asume it's OTel,
> > > > otherwise... assume it's something else.
> > > >
> > > > That's not unfeasible, but I wonder if it isn't better to store the 2
> > > > ids separately.
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
> > > > >
> > > > > Thanks Alex for raising this discussion. Thanks everyone for chiming
> > > in.
> > > > > The plan looks good to me overall.
> > > > >
> > > > > > Update PolarisEvent, expose request ID if available, expose OTel
> > > > context
> > > > > if available.
> > > > >
> > > > > https://github.com/apache/polaris/pull/2914 persists both request id
> > > and
> > > > > otel context.
> > > > >
> > > > > Do we actually need to persist both?
> > > > > From the original discussion, the intent of a request ID was to
> > > correlate
> > > > > multiple events generated by the same request, for example, linking a
> > > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense, what
> > we
> > > > > really need is a single opaque identifier that ties related events
> > > > > together, regardless of whether it comes from a request ID or an OTel
> > > > > context.
> > > > >
> > > > > If we persist both fields (requestId and otelContext), it can lead to
> > > > > consumer confusion, since downstream systems would need to decide
> > which
> > > > one
> > > > > to rely on, potentially causing inconsistent event correlation.
> > Unless
> > > > > there are concrete use cases where event consumers need both, I’d
> > > suggest
> > > > > persisting a single opaque correlation ID instead. This keeps the
> > > schema
> > > > > simpler and ensures consistent event linking.
> > > > >
> > > > > Yufei
> > > > >
> > > > >
> > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org>
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> > > jb@nanthrax.net>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi
> > > > > > >
> > > > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > > > >
> > > > > > > Thanks !
> > > > > > > Regards
> > > > > > > JB
> > > > > > >
> > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <adutra@apache.org
> > >
> > > a
> > > > > > écrit :
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Thanks for the good discussion so far. I think we have a broad
> > > > > > consensus.
> > > > > > > >
> > > > > > > > Unless anybody disagrees I am going to move forward with the
> > the
> > > > > > > > following revised plan, incorporating the latest feedback:
> > > > > > > >
> > > > > > > > 1. Restore the original functionality for request IDs, change
> > the
> > > > > > > > default header name back to x-request-id
> > > > > > > >
> > > > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > > > >
> > > > > > > > 3. Update PolarisEvent, expose request ID if available, expose
> > > OTel
> > > > > > > > context if available.
> > > > > > > >
> > > > > > > > 4. Update events table SQL schema: insert request ID if
> > > available,
> > > > > > > > insert OTel context if available.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > > > jb@nanthrax.net
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi
> > > > > > > > >
> > > > > > > > > I think it's better to use w3c_trace_context (better than two
> > > > > > fields).
> > > > > > > > >
> > > > > > > > > Regarding x-request-id, I think it's for a different purpose,
> > > > > > > > > specifically for persistence. Personally, I think we can
> > > achieve
> > > > the
> > > > > > > > > same with span and events.
> > > > > > > > > If we want to "distinguish" the two layers, it makes sense to
> > > > have
> > > > > > > > > x-request-id (not sure it's super helpful). Else, I think we
> > > can
> > > > > > > > > achieve the same with span/event.
> > > > > > > > >
> > > > > > > > > Just my $0.01
> > > > > > > > >
> > > > > > > > > Regards
> > > > > > > > > JB
> > > > > > > > >
> > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as
> > > > well.
> > > > > > Great
> > > > > > > > find, Dmitri!
> > > > > > > > > >
> > > > > > > > > > I’m still debating with myself on whether we need to store
> > > the
> > > > > > > > `x-request-id` field as part of the Events persistence. Can we
> > > > think
> > > > > > of a
> > > > > > > > good use case where this is more helpful to the user than the
> > > OTel
> > > > > > > > Trace/Span IDs? I am making the assumption here that those are
> > > > still
> > > > > > being
> > > > > > > > returned back to the client.
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Adnan Hemani
> > > > > > > > > >
> > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > > > adutra@apache.org>
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi Dmitri,
> > > > > > > > > > >
> > > > > > > > > > > Yes, I think your suggestion to use just one field
> > > > > > w3c_trace_context
> > > > > > > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > > > > > > >
> > > > > > > > > > > With that, I also think that we are slowly drifting into
> > > > > > > > > > > implementation considerations; let's get consensus on the
> > > > general
> > > > > > > > > > > design first, and we can certainly fine-tune the actual
> > > Java
> > > > > > methods
> > > > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Alex
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > > > > dimas@apache.org> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> Hi All,
> > > > > > > > > > >>
> > > > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > > > >>
> > > > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > > > >>
> > > > > > > > > > >> Re: (3) Can we abstract request/trace info into a
> > separate
> > > > > > object
> > > > > > > > without
> > > > > > > > > > >> exposing those accessors on the Event class directly?
> > OTel
> > > > > > defines
> > > > > > > > > > >> trace/span concepts, but in request ID is a bit foreign
> > to
> > > > OTel.
> > > > > > > > Having
> > > > > > > > > > >> tracing / request ID isolated in java could help with
> > > > > > maintaining
> > > > > > > > it and
> > > > > > > > > > >> potentially supporting other (custom) tracing methods.
> > > > > > > > > > >>
> > > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation
> > data
> > > > in the
> > > > > > > > form of a
> > > > > > > > > > >> standard context propagation string, e.g. W3C
> > > trace-context
> > > > [1]
> > > > > > > > (same value
> > > > > > > > > > >> as its HTTP header), so the column could be called
> > > > > > > > w3c_trace_context or
> > > > > > > > > > >> simply trace_context.
> > > > > > > > > > >>
> > > > > > > > > > >> Open question: do we need to write a separate,
> > individual
> > > > trace
> > > > > > ID
> > > > > > > > field in
> > > > > > > > > > >> SQL? I suppose it is not very useful since correlating
> > it
> > > to
> > > > > > other
> > > > > > > > trace
> > > > > > > > > > >> data already requires understanding OTel context
> > > propagation
> > > > > > and a
> > > > > > > > query
> > > > > > > > > > >> against trace_context can still be made using
> > > > string-matching
> > > > > > > > clauses. We
> > > > > > > > > > >> could probably (additionally) store it in the request_id
> > > > column
> > > > > > if
> > > > > > > > the
> > > > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > > > >>
> > > > > > > > > > >> As for span ID, I do not really see a use case for
> > > > persisting it
> > > > > > > > > > >> individually. It is very specific to OTel trace data
> > > > > > construction.
> > > > > > > > > > >>
> > > > > > > > > > >> Actually, using the W3C trace context [1] encoding
> > > probably
> > > > > > makes
> > > > > > > > sense in
> > > > > > > > > > >> the java event representation too. Interested callers
> > can
> > > > easily
> > > > > > > > decode
> > > > > > > > > > >> this information since the format is well-defined. As a
> > > side
> > > > > > > > benefit, this
> > > > > > > > > > >> opens opportunities for downstream event consumers to
> > > > connect
> > > > > > > > (propagate
> > > > > > > > > > >> context) to traces that produced events based on the
> > event
> > > > data
> > > > > > > > itself,
> > > > > > > > > > >> without relying on the intermediate frameworks. This may
> > > be
> > > > > > > > desirable since
> > > > > > > > > > >> the current Polaris event persistence impl. writes
> > events
> > > in
> > > > > > > > batches, so
> > > > > > > > > > >> the association to individual requests that produced
> > those
> > > > > > events
> > > > > > > > is lost.
> > > > > > > > > > >> Whether to perform this kind of context propagation will
> > > be
> > > > at
> > > > > > the
> > > > > > > > > > >> discretion of the event consumer, of course (outside of
> > > > Polaris
> > > > > > > > code).
> > > > > > > > > > >>
> > > > > > > > > > >> [1]
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > > > >>
> > > > > > > > > > >> WDYT?
> > > > > > > > > > >>
> > > > > > > > > > >> Thanks,
> > > > > > > > > > >> Dmitri.
> > > > > > > > > > >>
> > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > > > > adutra@apache.org>
> > > > > > > > wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>> Hi all,
> > > > > > > > > > >>>
> > > > > > > > > > >>> Thank you for chiming in; the context around request
> > IDs
> > > > is now
> > > > > > > > clear.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Trying to summarize this thread into actionable items,
> > > > here's
> > > > > > what
> > > > > > > > I
> > > > > > > > > > >>> propose:
> > > > > > > > > > >>>
> > > > > > > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > > > > > > >>> - Change the default header name back to
> > x-request-id
> > > > > > (despite
> > > > > > > > the
> > > > > > > > > > >>> x- prefix being deprecated), but keep it configurable
> > as
> > > > today.
> > > > > > > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > > > >>> - Do not generate a request ID if the REST client
> > > > doesn't
> > > > > > > > provide one.
> > > > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > > > >>> - Expose new requestId(), traceId(), spanId()
> > methods,
> > > > all
> > > > > > > > nullable.
> > > > > > > > > > >>> - This would align with the emerging consensus
> > around
> > > > > > including
> > > > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > > > >>> - Insert the client-provided request ID into the
> > > > request_id
> > > > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > > > >>> - Add two new nullable columns, trace_id and
> > span_id,
> > > > and
> > > > > > > > populate
> > > > > > > > > > >>> them if OTel is enabled.
> > > > > > > > > > >>>
> > > > > > > > > > >>> From our discussions, I think it's important to not
> > > > conflate
> > > > > > OTel
> > > > > > > > > > >>> tracing fields with Envoy tracing fields, which is why
> > I
> > > > > > suggest we
> > > > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > > > >>>
> > > > > > > > > > >>> Thanks,
> > > > > > > > > > >>> Alex
> > > > > > > > > > >>>
> > > > > > > > > > >>> [1]:
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> Hi all,
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> Thanks to Alex for starting this thread - because of
> > > > this, I’m
> > > > > > > > just
> > > > > > > > > > >>> coming to the realization that OTel Trace and Span IDs
> > > are
> > > > > > coming
> > > > > > > > built-in
> > > > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > > > Request/Correlation ID is
> > > > > > > > > > >>> likely not needed as a result. My original motivation
> > for
> > > > > > > > generation of a
> > > > > > > > > > >>> Request/Correlation ID was to ensure that any client
> > can
> > > > > > uniquely
> > > > > > > > identify
> > > > > > > > > > >>> a request made to Polaris, which would be especially
> > > > useful for
> > > > > > > > debugging
> > > > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should
> > > > remove
> > > > > > the
> > > > > > > > > > >>> Request/Correlation ID generation and always use the
> > OTel
> > > > > > > > trace/span IDs
> > > > > > > > > > >>> (which come for free with Quarkus) instead for the
> > > > Correlation
> > > > > > ID
> > > > > > > > unless a
> > > > > > > > > > >>> valid header is present, which would take over as the
> > > > > > Correlation
> > > > > > > > ID
> > > > > > > > > > >>> instead.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> —
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> > > > intended
> > > > > > use
> > > > > > > > case is
> > > > > > > > > > >>> to provide some sort of correlation between events that
> > > > have
> > > > > > > > occurred as
> > > > > > > > > > >>> part of the same request. For example, if a user makes
> > an
> > > > > > > > CommitTransaction
> > > > > > > > > > >>> request, it would be helpful to see all UpdateTable
> > calls
> > > > that
> > > > > > > > were made as
> > > > > > > > > > >>> part of that one user request.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> Best,
> > > > > > > > > > >>>> Adnan Hemani
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > > > > > > dimas@apache.org>
> > > > > > > > > > >>> wrote:
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> Hi Michael,
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> Just to confirm: if / when we restore that, Polaris
> > > will
> > > > NOT
> > > > > > > > generate
> > > > > > > > > > >>> new
> > > > > > > > > > >>>>> IDs in case the header is not present in the request,
> > > > > > correct?
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> What about adding request IDs to events [1][2]?
> > What's
> > > > the
> > > > > > > > intended use
> > > > > > > > > > >>>>> case for that? Could you share some context here too?
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event
> > persistence
> > > > as
> > > > > > > > "beta" in
> > > > > > > > > > >>>>> 1.2.0... This discussion adds another point towards
> > > > that, I
> > > > > > > > think.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> [1]
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > > > >>>>> [2]
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> [2877]
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> Thanks,
> > > > > > > > > > >>>>> Dmitri.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > >>>>> wrote:
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>> The generating a request id is new code that was
> > added
> > > > > > after the
> > > > > > > > > > >>> original
> > > > > > > > > > >>>>>> x-request-id support. You can see the state from ~1
> > > year
> > > > > > ago, we
> > > > > > > > > > >>> hard-coded
> > > > > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > > > >>>>>> . At some point, it was changed to be configurable,
> > > > then the
> > > > > > > > > > >>>>>> ContextResolverFilter filter was
> > refactored/eliminated
> > > > and
> > > > > > the
> > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but lost
> > some
> > > > of
> > > > > > its
> > > > > > > > original
> > > > > > > > > > >>>>>> functionality.
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>> My personal opinion is that restoring support for
> > the
> > > > > > > > x-request-id
> > > > > > > > > > >>> header
> > > > > > > > > > >>>>>> is something that we should do, but if the header
> > > isn't
> > > > > > present,
> > > > > > > > > > >>> falling
> > > > > > > > > > >>>>>> back on simply using OTel trace ids is good enough
> > > > (better,
> > > > > > > > even) than
> > > > > > > > > > >>>>>> generating another random request id.
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>> Mike
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> > Bourlatchkov <
> > > > > > > > > > >>> dimas@apache.org>
> > > > > > > > > > >>>>>> wrote:
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes sense to
> > > me.
> > > > > > > > However, I
> > > > > > > > > > >>>>>> wonder:
> > > > > > > > > > >>>>>>> why would Polaris need to generate a new request ID
> > > > inside
> > > > > > its
> > > > > > > > > > >>> code?..
> > > > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> How important is it to propagate this ID to Polaris
> > > > Events?
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> Thanks,
> > > > > > > > > > >>>>>>> Dmitri.
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > > >>>>>>> wrote:
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>>> I think the original intention for this requestId
> > > > field
> > > > > > was to
> > > > > > > > > > >>> support
> > > > > > > > > > >>>>>>>> request propagation from load balancers, like
> > Envoy
> > > (
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why
> > the
> > > > > > default
> > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally
> > a
> > > > custom
> > > > > > > > thing,
> > > > > > > > > > >>> but
> > > > > > > > > > >>>>>>> then
> > > > > > > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > > > > > > Unfortunately,
> > > > > > > > > > >>>>>> looking
> > > > > > > > > > >>>>>>>> through the code, I think that the actual
> > functional
> > > > > > plumbing
> > > > > > > > has
> > > > > > > > > > >>> been
> > > > > > > > > > >>>>>>> lost
> > > > > > > > > > >>>>>>>> in the course of multiple refactors around the
> > call
> > > > > > context
> > > > > > > > and
> > > > > > > > > > >>>>>>> resolver. I
> > > > > > > > > > >>>>>>>> don't see references to that property or the
> > > > underlying
> > > > > > > > header.
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header
> > feels
> > > > like
> > > > > > > > something
> > > > > > > > > > >>> we
> > > > > > > > > > >>>>>>>> should definitely keep, especially when Polaris is
> > > one
> > > > > > > > service in a
> > > > > > > > > > >>>>>> mesh
> > > > > > > > > > >>>>>>> of
> > > > > > > > > > >>>>>>>> services that maybe don't have OTel integration.
> > > I'm a
> > > > > > fan of
> > > > > > > > the
> > > > > > > > > > >>> OTel
> > > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and
> > there
> > > > are
> > > > > > many
> > > > > > > > > > >>>>>> middleware
> > > > > > > > > > >>>>>>>> layers that know how to forward on the
> > x-request-id
> > > > > > header.
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>> Mike
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > > > > snazy@snazy.de>
> > > > > > > > > > >>> wrote:
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with the
> > > > existing
> > > > > > > > de-facto
> > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to
> > > integrate
> > > > > > into
> > > > > > > > their
> > > > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > > > >>>>>>>>>
> > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> > > Bourlatchkov <
> > > > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> I certainly support the idea of following OTel
> > > > > > standards for
> > > > > > > > > > >>>>>>> achieving
> > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or
> > events.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> As to what form the correlation data should
> > take,
> > > I
> > > > > > believe
> > > > > > > > it is
> > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context" represents.
> > > > So, I
> > > > > > > > believe it
> > > > > > > > > > >>>>>>> makes
> > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard context
> > > > > > propagators
> > > > > > > > at the
> > > > > > > > > > >>>>>> API
> > > > > > > > > > >>>>>>>>> layer.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > > > information,
> > > > > > then
> > > > > > > > > > >>>>>> returning
> > > > > > > > > > >>>>>>>> any
> > > > > > > > > > >>>>>>>>>> other "correlation" data in the response is
> > > > redundant, I
> > > > > > > > think.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel
> > context
> > > > info,
> > > > > > > > what is
> > > > > > > > > > >>>>>> the
> > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > > > "correlation
> > > > > > ID"?
> > > > > > > > How is
> > > > > > > > > > >>>>>> it
> > > > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris
> > > response
> > > > > > > > (operation)
> > > > > > > > > > >>>>>> with
> > > > > > > > > > >>>>>>>>> events
> > > > > > > > > > >>>>>>>>>> that might have resulted from its execution, I
> > > > believe a
> > > > > > > > more
> > > > > > > > > > >>>>>> robust
> > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel trace
> > info
> > > > > > > > (starting a new
> > > > > > > > > > >>>>>>>> trace
> > > > > > > > > > >>>>>>>>> if
> > > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can
> > also
> > > > > > return
> > > > > > > > the
> > > > > > > > > > >>> trace
> > > > > > > > > > >>>>>>>>> context
> > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit
> > awkward
> > > > from
> > > > > > the
> > > > > > > > OTel
> > > > > > > > > > >>>>>>>>>> perspective, but might be an option for
> > supporting
> > > > > > custom
> > > > > > > > > > >>>>>>> correlators.
> > > > > > > > > > >>>>>>>> It
> > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The header
> > > name
> > > > > > could be
> > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to extract
> > > the
> > > > > > trace
> > > > > > > > ID from
> > > > > > > > > > >>>>>> the
> > > > > > > > > > >>>>>>>>>> response and from events and find related data.
> > > > > > Granted, it
> > > > > > > > will
> > > > > > > > > > >>>>>>>> require
> > > > > > > > > > >>>>>>>>> a
> > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to decode
> > > trace
> > > > IDs
> > > > > > > > from the
> > > > > > > > > > >>>>>> OTel
> > > > > > > > > > >>>>>>>>>> context, but the format is standard and not
> > > > complex. The
> > > > > > > > benefit
> > > > > > > > > > >>>>>> for
> > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate
> > > > with
> > > > > > > > > > >>>>>> OTel-compatible
> > > > > > > > > > >>>>>>>>>> observability platforms regardless of whether
> > any
> > > > > > particular
> > > > > > > > > > >>>>>>> deployment
> > > > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra
> > <
> > > > > > > > > > >>> adutra@apache.org
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID",
> > > but
> > > > its
> > > > > > > > purpose is
> > > > > > > > > > >>>>>>> not
> > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > > > observability
> > > > > > > > feature to
> > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to
> > > > rename it
> > > > > > to
> > > > > > > > > > >>>>>>>> "correlation
> > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry
> > standards
> > > > [1].
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light overlaps
> > > with
> > > > > > core
> > > > > > > > > > >>>>>> telemetry
> > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled
> > in
> > > > > > Polaris,
> > > > > > > > each
> > > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID,
> > > making
> > > > a
> > > > > > > > separate
> > > > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID
> > in
> > > > > > Polaris
> > > > > > > > events,
> > > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > > > > > > significantly
> > > > > > > > > > >>>>>>>> simplify
> > > > > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and
> > span
> > > > ID as
> > > > > > the
> > > > > > > > > > >>>>>>>> correlation
> > > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a
> > > random
> > > > > > > > correlation
> > > > > > > > > > >>>>>> ID.
> > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> > correlation
> > > > ID
> > > > > > > > header is
> > > > > > > > > > >>>>>>>> present
> > > > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is met,
> > > > generate
> > > > > > a
> > > > > > > > random
> > > > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best approach
> > > when a
> > > > > > > > correlation
> > > > > > > > > > >>>>>> ID
> > > > > > > > > > >>>>>>>>>>> header is present in the request. However, I
> > > > believe it
> > > > > > > > would be
> > > > > > > > > > >>>>>>> more
> > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as
> > > > OTel
> > > > > > > > offers a
> > > > > > > > > > >>>>>> more
> > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> > > > propagation,
> > > > > > > > e.g. W3C
> > > > > > > > > > >>>>>>>> Trace
> > > > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > >>>>>>>>>
> > > > > > > > > > >>>>>>>>
> > > > > > > > > > >>>>>>>
> > > > > > > > > > >>>>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Yufei,
[...] tying a BeforeUpdateTable event to the corresponding AfterUpdateTable
event.
It looks like this calls for a strong and always present ID in events.This
ID is obviously distincts from OTel context information and should be
persisted separately, IMHO.
I believe there's still an open question whether the request ID in the
events should be populated from the x-request-id in REST API headers.
Yufei: Is that required in your use cases?
Thanks,
Dmitri.
On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <fl...@gmail.com> wrote:
> I think the first question is whether any downstream use cases need to
> interpret the semantics of a request ID or OTel context. As I mentioned,
> the original purpose of a request ID was event correlation, e.g., tying a
> BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
> that’s the only requirement, an opaque correlation ID is good enough. We
> could change the field name(request-id) if needed.
>
> If we do have use cases that depend on the meaning of the request ID or
> OTel context, we can store them in additional_properties (
>
> https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134
> ).
> This is more flexible because it allows us to:
> 1. Record a key that indicates the format (e.g., x-request-id vs.
> otel-context).
> 2. Preserve both when Polaris receives both headers.
> 3. Add future tracing fields without a schema change.
>
> Yufei
>
>
> On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
> > Hi All,
> >
> > From my POV the use cases for x-request-id and OTel are slightly
> different.
> > OTel is an observability use case where cross-linking of various pieces
> of
> > data is desired but not required in all cases (data loss is acceptable).
> > The x-request-id use case appears to have some very specific custom
> > applications without any obvious relationship to OTel.
> >
> > Given the current state of this discussion, I think it makes sense to
> > handle and persist them separately.
> >
> > However, if we intend x-request-id to act as an observability aid
> > alternative to OTel, then it might be preferable to have a config flag
> > indicating which one of them should be added to persistent event data. In
> > that case we should reuse the same column for them (and possibly add a
> > "type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...")
> >
> > WDYT?
> >
> > Thanks,
> > Dmitri.
> >
> >
> > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> > > Hi Yufei,
> > >
> > > > I’d suggest persisting a single opaque correlation ID instead
> > >
> > > The problem I see with this is that readers cannot know the syntax of
> > > the opaque id, and will have to "probe" using some heuristics, e.g. if
> > > the id matches the OTel trace context syntax, asume it's OTel,
> > > otherwise... assume it's something else.
> > >
> > > That's not unfeasible, but I wonder if it isn't better to store the 2
> > > ids separately.
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
> > > >
> > > > Thanks Alex for raising this discussion. Thanks everyone for chiming
> > in.
> > > > The plan looks good to me overall.
> > > >
> > > > > Update PolarisEvent, expose request ID if available, expose OTel
> > > context
> > > > if available.
> > > >
> > > > https://github.com/apache/polaris/pull/2914 persists both request id
> > and
> > > > otel context.
> > > >
> > > > Do we actually need to persist both?
> > > > From the original discussion, the intent of a request ID was to
> > correlate
> > > > multiple events generated by the same request, for example, linking a
> > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense, what
> we
> > > > really need is a single opaque identifier that ties related events
> > > > together, regardless of whether it comes from a request ID or an OTel
> > > > context.
> > > >
> > > > If we persist both fields (requestId and otelContext), it can lead to
> > > > consumer confusion, since downstream systems would need to decide
> which
> > > one
> > > > to rely on, potentially causing inconsistent event correlation.
> Unless
> > > > there are concrete use cases where event consumers need both, I’d
> > suggest
> > > > persisting a single opaque correlation ID instead. This keeps the
> > schema
> > > > simpler and ensures consistent event linking.
> > > >
> > > > Yufei
> > > >
> > > >
> > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> > jb@nanthrax.net>
> > > > > wrote:
> > > > > >
> > > > > > Hi
> > > > > >
> > > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > > >
> > > > > > Thanks !
> > > > > > Regards
> > > > > > JB
> > > > > >
> > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <adutra@apache.org
> >
> > a
> > > > > écrit :
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Thanks for the good discussion so far. I think we have a broad
> > > > > consensus.
> > > > > > >
> > > > > > > Unless anybody disagrees I am going to move forward with the
> the
> > > > > > > following revised plan, incorporating the latest feedback:
> > > > > > >
> > > > > > > 1. Restore the original functionality for request IDs, change
> the
> > > > > > > default header name back to x-request-id
> > > > > > >
> > > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > > >
> > > > > > > 3. Update PolarisEvent, expose request ID if available, expose
> > OTel
> > > > > > > context if available.
> > > > > > >
> > > > > > > 4. Update events table SQL schema: insert request ID if
> > available,
> > > > > > > insert OTel context if available.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > > jb@nanthrax.net
> > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi
> > > > > > > >
> > > > > > > > I think it's better to use w3c_trace_context (better than two
> > > > > fields).
> > > > > > > >
> > > > > > > > Regarding x-request-id, I think it's for a different purpose,
> > > > > > > > specifically for persistence. Personally, I think we can
> > achieve
> > > the
> > > > > > > > same with span and events.
> > > > > > > > If we want to "distinguish" the two layers, it makes sense to
> > > have
> > > > > > > > x-request-id (not sure it's super helpful). Else, I think we
> > can
> > > > > > > > achieve the same with span/event.
> > > > > > > >
> > > > > > > > Just my $0.01
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > JB
> > > > > > > >
> > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as
> > > well.
> > > > > Great
> > > > > > > find, Dmitri!
> > > > > > > > >
> > > > > > > > > I’m still debating with myself on whether we need to store
> > the
> > > > > > > `x-request-id` field as part of the Events persistence. Can we
> > > think
> > > > > of a
> > > > > > > good use case where this is more helpful to the user than the
> > OTel
> > > > > > > Trace/Span IDs? I am making the assumption here that those are
> > > still
> > > > > being
> > > > > > > returned back to the client.
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Adnan Hemani
> > > > > > > > >
> > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > > adutra@apache.org>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Dmitri,
> > > > > > > > > >
> > > > > > > > > > Yes, I think your suggestion to use just one field
> > > > > w3c_trace_context
> > > > > > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > > > > > >
> > > > > > > > > > With that, I also think that we are slowly drifting into
> > > > > > > > > > implementation considerations; let's get consensus on the
> > > general
> > > > > > > > > > design first, and we can certainly fine-tune the actual
> > Java
> > > > > methods
> > > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Alex
> > > > > > > > > >
> > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > > > dimas@apache.org> wrote:
> > > > > > > > > >>
> > > > > > > > > >> Hi All,
> > > > > > > > > >>
> > > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > > >>
> > > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > > >>
> > > > > > > > > >> Re: (3) Can we abstract request/trace info into a
> separate
> > > > > object
> > > > > > > without
> > > > > > > > > >> exposing those accessors on the Event class directly?
> OTel
> > > > > defines
> > > > > > > > > >> trace/span concepts, but in request ID is a bit foreign
> to
> > > OTel.
> > > > > > > Having
> > > > > > > > > >> tracing / request ID isolated in java could help with
> > > > > maintaining
> > > > > > > it and
> > > > > > > > > >> potentially supporting other (custom) tracing methods.
> > > > > > > > > >>
> > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation
> data
> > > in the
> > > > > > > form of a
> > > > > > > > > >> standard context propagation string, e.g. W3C
> > trace-context
> > > [1]
> > > > > > > (same value
> > > > > > > > > >> as its HTTP header), so the column could be called
> > > > > > > w3c_trace_context or
> > > > > > > > > >> simply trace_context.
> > > > > > > > > >>
> > > > > > > > > >> Open question: do we need to write a separate,
> individual
> > > trace
> > > > > ID
> > > > > > > field in
> > > > > > > > > >> SQL? I suppose it is not very useful since correlating
> it
> > to
> > > > > other
> > > > > > > trace
> > > > > > > > > >> data already requires understanding OTel context
> > propagation
> > > > > and a
> > > > > > > query
> > > > > > > > > >> against trace_context can still be made using
> > > string-matching
> > > > > > > clauses. We
> > > > > > > > > >> could probably (additionally) store it in the request_id
> > > column
> > > > > if
> > > > > > > the
> > > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > > >>
> > > > > > > > > >> As for span ID, I do not really see a use case for
> > > persisting it
> > > > > > > > > >> individually. It is very specific to OTel trace data
> > > > > construction.
> > > > > > > > > >>
> > > > > > > > > >> Actually, using the W3C trace context [1] encoding
> > probably
> > > > > makes
> > > > > > > sense in
> > > > > > > > > >> the java event representation too. Interested callers
> can
> > > easily
> > > > > > > decode
> > > > > > > > > >> this information since the format is well-defined. As a
> > side
> > > > > > > benefit, this
> > > > > > > > > >> opens opportunities for downstream event consumers to
> > > connect
> > > > > > > (propagate
> > > > > > > > > >> context) to traces that produced events based on the
> event
> > > data
> > > > > > > itself,
> > > > > > > > > >> without relying on the intermediate frameworks. This may
> > be
> > > > > > > desirable since
> > > > > > > > > >> the current Polaris event persistence impl. writes
> events
> > in
> > > > > > > batches, so
> > > > > > > > > >> the association to individual requests that produced
> those
> > > > > events
> > > > > > > is lost.
> > > > > > > > > >> Whether to perform this kind of context propagation will
> > be
> > > at
> > > > > the
> > > > > > > > > >> discretion of the event consumer, of course (outside of
> > > Polaris
> > > > > > > code).
> > > > > > > > > >>
> > > > > > > > > >> [1]
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > > >>
> > > > > > > > > >> WDYT?
> > > > > > > > > >>
> > > > > > > > > >> Thanks,
> > > > > > > > > >> Dmitri.
> > > > > > > > > >>
> > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > > > adutra@apache.org>
> > > > > > > wrote:
> > > > > > > > > >>
> > > > > > > > > >>> Hi all,
> > > > > > > > > >>>
> > > > > > > > > >>> Thank you for chiming in; the context around request
> IDs
> > > is now
> > > > > > > clear.
> > > > > > > > > >>>
> > > > > > > > > >>> Trying to summarize this thread into actionable items,
> > > here's
> > > > > what
> > > > > > > I
> > > > > > > > > >>> propose:
> > > > > > > > > >>>
> > > > > > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > > > > > >>> - Change the default header name back to
> x-request-id
> > > > > (despite
> > > > > > > the
> > > > > > > > > >>> x- prefix being deprecated), but keep it configurable
> as
> > > today.
> > > > > > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > > >>> - Do not generate a request ID if the REST client
> > > doesn't
> > > > > > > provide one.
> > > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > > >>> - Expose new requestId(), traceId(), spanId()
> methods,
> > > all
> > > > > > > nullable.
> > > > > > > > > >>> - This would align with the emerging consensus
> around
> > > > > including
> > > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > > >>> - Insert the client-provided request ID into the
> > > request_id
> > > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > > >>> - Add two new nullable columns, trace_id and
> span_id,
> > > and
> > > > > > > populate
> > > > > > > > > >>> them if OTel is enabled.
> > > > > > > > > >>>
> > > > > > > > > >>> From our discussions, I think it's important to not
> > > conflate
> > > > > OTel
> > > > > > > > > >>> tracing fields with Envoy tracing fields, which is why
> I
> > > > > suggest we
> > > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > > >>>
> > > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > > >>>
> > > > > > > > > >>> Thanks,
> > > > > > > > > >>> Alex
> > > > > > > > > >>>
> > > > > > > > > >>> [1]:
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> Hi all,
> > > > > > > > > >>>>
> > > > > > > > > >>>> Thanks to Alex for starting this thread - because of
> > > this, I’m
> > > > > > > just
> > > > > > > > > >>> coming to the realization that OTel Trace and Span IDs
> > are
> > > > > coming
> > > > > > > built-in
> > > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > > Request/Correlation ID is
> > > > > > > > > >>> likely not needed as a result. My original motivation
> for
> > > > > > > generation of a
> > > > > > > > > >>> Request/Correlation ID was to ensure that any client
> can
> > > > > uniquely
> > > > > > > identify
> > > > > > > > > >>> a request made to Polaris, which would be especially
> > > useful for
> > > > > > > debugging
> > > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > > >>>>
> > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should
> > > remove
> > > > > the
> > > > > > > > > >>> Request/Correlation ID generation and always use the
> OTel
> > > > > > > trace/span IDs
> > > > > > > > > >>> (which come for free with Quarkus) instead for the
> > > Correlation
> > > > > ID
> > > > > > > unless a
> > > > > > > > > >>> valid header is present, which would take over as the
> > > > > Correlation
> > > > > > > ID
> > > > > > > > > >>> instead.
> > > > > > > > > >>>>
> > > > > > > > > >>>> —
> > > > > > > > > >>>>
> > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> > > intended
> > > > > use
> > > > > > > case is
> > > > > > > > > >>> to provide some sort of correlation between events that
> > > have
> > > > > > > occurred as
> > > > > > > > > >>> part of the same request. For example, if a user makes
> an
> > > > > > > CommitTransaction
> > > > > > > > > >>> request, it would be helpful to see all UpdateTable
> calls
> > > that
> > > > > > > were made as
> > > > > > > > > >>> part of that one user request.
> > > > > > > > > >>>>
> > > > > > > > > >>>> Best,
> > > > > > > > > >>>> Adnan Hemani
> > > > > > > > > >>>>
> > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > > > > > dimas@apache.org>
> > > > > > > > > >>> wrote:
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Hi Michael,
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Just to confirm: if / when we restore that, Polaris
> > will
> > > NOT
> > > > > > > generate
> > > > > > > > > >>> new
> > > > > > > > > >>>>> IDs in case the header is not present in the request,
> > > > > correct?
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> What about adding request IDs to events [1][2]?
> What's
> > > the
> > > > > > > intended use
> > > > > > > > > >>>>> case for that? Could you share some context here too?
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Side note: I proposed [2877] flagging event
> persistence
> > > as
> > > > > > > "beta" in
> > > > > > > > > >>>>> 1.2.0... This discussion adds another point towards
> > > that, I
> > > > > > > think.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> [1]
> > > > > > > > > >>>>>
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > > >>>>> [2]
> > > > > > > > > >>>>>
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> [2877]
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Thanks,
> > > > > > > > > >>>>> Dmitri.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > >>>>> wrote:
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>> The generating a request id is new code that was
> added
> > > > > after the
> > > > > > > > > >>> original
> > > > > > > > > >>>>>> x-request-id support. You can see the state from ~1
> > year
> > > > > ago, we
> > > > > > > > > >>> hard-coded
> > > > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>>
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > > >>>>>> . At some point, it was changed to be configurable,
> > > then the
> > > > > > > > > >>>>>> ContextResolverFilter filter was
> refactored/eliminated
> > > and
> > > > > the
> > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but lost
> some
> > > of
> > > > > its
> > > > > > > original
> > > > > > > > > >>>>>> functionality.
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>> My personal opinion is that restoring support for
> the
> > > > > > > x-request-id
> > > > > > > > > >>> header
> > > > > > > > > >>>>>> is something that we should do, but if the header
> > isn't
> > > > > present,
> > > > > > > > > >>> falling
> > > > > > > > > >>>>>> back on simply using OTel trace ids is good enough
> > > (better,
> > > > > > > even) than
> > > > > > > > > >>>>>> generating another random request id.
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>> Mike
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri
> Bourlatchkov <
> > > > > > > > > >>> dimas@apache.org>
> > > > > > > > > >>>>>> wrote:
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes sense to
> > me.
> > > > > > > However, I
> > > > > > > > > >>>>>> wonder:
> > > > > > > > > >>>>>>> why would Polaris need to generate a new request ID
> > > inside
> > > > > its
> > > > > > > > > >>> code?..
> > > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> How important is it to propagate this ID to Polaris
> > > Events?
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> Thanks,
> > > > > > > > > >>>>>>> Dmitri.
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > > >>>>>>> wrote:
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>>> I think the original intention for this requestId
> > > field
> > > > > was to
> > > > > > > > > >>> support
> > > > > > > > > >>>>>>>> request propagation from load balancers, like
> Envoy
> > (
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why
> the
> > > > > default
> > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally
> a
> > > custom
> > > > > > > thing,
> > > > > > > > > >>> but
> > > > > > > > > >>>>>>> then
> > > > > > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > > > > > Unfortunately,
> > > > > > > > > >>>>>> looking
> > > > > > > > > >>>>>>>> through the code, I think that the actual
> functional
> > > > > plumbing
> > > > > > > has
> > > > > > > > > >>> been
> > > > > > > > > >>>>>>> lost
> > > > > > > > > >>>>>>>> in the course of multiple refactors around the
> call
> > > > > context
> > > > > > > and
> > > > > > > > > >>>>>>> resolver. I
> > > > > > > > > >>>>>>>> don't see references to that property or the
> > > underlying
> > > > > > > header.
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header
> feels
> > > like
> > > > > > > something
> > > > > > > > > >>> we
> > > > > > > > > >>>>>>>> should definitely keep, especially when Polaris is
> > one
> > > > > > > service in a
> > > > > > > > > >>>>>> mesh
> > > > > > > > > >>>>>>> of
> > > > > > > > > >>>>>>>> services that maybe don't have OTel integration.
> > I'm a
> > > > > fan of
> > > > > > > the
> > > > > > > > > >>> OTel
> > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and
> there
> > > are
> > > > > many
> > > > > > > > > >>>>>> middleware
> > > > > > > > > >>>>>>>> layers that know how to forward on the
> x-request-id
> > > > > header.
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>> Mike
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > > > snazy@snazy.de>
> > > > > > > > > >>> wrote:
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with the
> > > existing
> > > > > > > de-facto
> > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to
> > integrate
> > > > > into
> > > > > > > their
> > > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > > >>>>>>>>>
> > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> > Bourlatchkov <
> > > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> I certainly support the idea of following OTel
> > > > > standards for
> > > > > > > > > >>>>>>> achieving
> > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or
> events.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> As to what form the correlation data should
> take,
> > I
> > > > > believe
> > > > > > > it is
> > > > > > > > > >>>>>>>>>> conceptually what the OTel "context" represents.
> > > So, I
> > > > > > > believe it
> > > > > > > > > >>>>>>> makes
> > > > > > > > > >>>>>>>>>> sense for Polaris to support standard context
> > > > > propagators
> > > > > > > at the
> > > > > > > > > >>>>>> API
> > > > > > > > > >>>>>>>>> layer.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > > information,
> > > > > then
> > > > > > > > > >>>>>> returning
> > > > > > > > > >>>>>>>> any
> > > > > > > > > >>>>>>>>>> other "correlation" data in the response is
> > > redundant, I
> > > > > > > think.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel
> context
> > > info,
> > > > > > > what is
> > > > > > > > > >>>>>> the
> > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > > "correlation
> > > > > ID"?
> > > > > > > How is
> > > > > > > > > >>>>>> it
> > > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris
> > response
> > > > > > > (operation)
> > > > > > > > > >>>>>> with
> > > > > > > > > >>>>>>>>> events
> > > > > > > > > >>>>>>>>>> that might have resulted from its execution, I
> > > believe a
> > > > > > > more
> > > > > > > > > >>>>>> robust
> > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel trace
> info
> > > > > > > (starting a new
> > > > > > > > > >>>>>>>> trace
> > > > > > > > > >>>>>>>>> if
> > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can
> also
> > > > > return
> > > > > > > the
> > > > > > > > > >>> trace
> > > > > > > > > >>>>>>>>> context
> > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit
> awkward
> > > from
> > > > > the
> > > > > > > OTel
> > > > > > > > > >>>>>>>>>> perspective, but might be an option for
> supporting
> > > > > custom
> > > > > > > > > >>>>>>> correlators.
> > > > > > > > > >>>>>>>> It
> > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The header
> > name
> > > > > could be
> > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> Custom correlation code will be able to extract
> > the
> > > > > trace
> > > > > > > ID from
> > > > > > > > > >>>>>> the
> > > > > > > > > >>>>>>>>>> response and from events and find related data.
> > > > > Granted, it
> > > > > > > will
> > > > > > > > > >>>>>>>> require
> > > > > > > > > >>>>>>>>> a
> > > > > > > > > >>>>>>>>>> bit more effort for the custom code to decode
> > trace
> > > IDs
> > > > > > > from the
> > > > > > > > > >>>>>> OTel
> > > > > > > > > >>>>>>>>>> context, but the format is standard and not
> > > complex. The
> > > > > > > benefit
> > > > > > > > > >>>>>> for
> > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate
> > > with
> > > > > > > > > >>>>>> OTel-compatible
> > > > > > > > > >>>>>>>>>> observability platforms regardless of whether
> any
> > > > > particular
> > > > > > > > > >>>>>>> deployment
> > > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra
> <
> > > > > > > > > >>> adutra@apache.org
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>>>> wrote:
> > > > > > > > > >>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID",
> > but
> > > its
> > > > > > > purpose is
> > > > > > > > > >>>>>>> not
> > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > > observability
> > > > > > > feature to
> > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to
> > > rename it
> > > > > to
> > > > > > > > > >>>>>>>> "correlation
> > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry
> standards
> > > [1].
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> However, this PR has brought to light overlaps
> > with
> > > > > core
> > > > > > > > > >>>>>> telemetry
> > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled
> in
> > > > > Polaris,
> > > > > > > each
> > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID,
> > making
> > > a
> > > > > > > separate
> > > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID
> in
> > > > > Polaris
> > > > > > > events,
> > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > > > > > significantly
> > > > > > > > > >>>>>>>> simplify
> > > > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and
> span
> > > ID as
> > > > > the
> > > > > > > > > >>>>>>>> correlation
> > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a
> > random
> > > > > > > correlation
> > > > > > > > > >>>>>> ID.
> > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific)
> correlation
> > > ID
> > > > > > > header is
> > > > > > > > > >>>>>>>> present
> > > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is met,
> > > generate
> > > > > a
> > > > > > > random
> > > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best approach
> > when a
> > > > > > > correlation
> > > > > > > > > >>>>>> ID
> > > > > > > > > >>>>>>>>>>> header is present in the request. However, I
> > > believe it
> > > > > > > would be
> > > > > > > > > >>>>>>> more
> > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as
> > > OTel
> > > > > > > offers a
> > > > > > > > > >>>>>> more
> > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> > > propagation,
> > > > > > > e.g. W3C
> > > > > > > > > >>>>>>>> Trace
> > > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > > >>>
> > > > > > >
> > > > >
> > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > > >>>>>>>>>>>
> > > > > > > > > >>>>>>>>>
> > > > > > > > > >>>>>>>>
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Yufei Gu <fl...@gmail.com>.
I think the first question is whether any downstream use cases need to
interpret the semantics of a request ID or OTel context. As I mentioned,
the original purpose of a request ID was event correlation, e.g., tying a
BeforeUpdateTable event to the corresponding AfterUpdateTable event. If
that’s the only requirement, an opaque correlation ID is good enough. We
could change the field name(request-id) if needed.
If we do have use cases that depend on the meaning of the request ID or
OTel context, we can store them in additional_properties (
https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134).
This is more flexible because it allows us to:
1. Record a key that indicates the format (e.g., x-request-id vs.
otel-context).
2. Preserve both when Polaris receives both headers.
3. Add future tracing fields without a schema change.
Yufei
On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <di...@apache.org>
wrote:
> Hi All,
>
> From my POV the use cases for x-request-id and OTel are slightly different.
> OTel is an observability use case where cross-linking of various pieces of
> data is desired but not required in all cases (data loss is acceptable).
> The x-request-id use case appears to have some very specific custom
> applications without any obvious relationship to OTel.
>
> Given the current state of this discussion, I think it makes sense to
> handle and persist them separately.
>
> However, if we intend x-request-id to act as an observability aid
> alternative to OTel, then it might be preferable to have a config flag
> indicating which one of them should be added to persistent event data. In
> that case we should reuse the same column for them (and possibly add a
> "type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...")
>
> WDYT?
>
> Thanks,
> Dmitri.
>
>
> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hi Yufei,
> >
> > > I’d suggest persisting a single opaque correlation ID instead
> >
> > The problem I see with this is that readers cannot know the syntax of
> > the opaque id, and will have to "probe" using some heuristics, e.g. if
> > the id matches the OTel trace context syntax, asume it's OTel,
> > otherwise... assume it's something else.
> >
> > That's not unfeasible, but I wonder if it isn't better to store the 2
> > ids separately.
> >
> > Thanks,
> > Alex
> >
> > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
> > >
> > > Thanks Alex for raising this discussion. Thanks everyone for chiming
> in.
> > > The plan looks good to me overall.
> > >
> > > > Update PolarisEvent, expose request ID if available, expose OTel
> > context
> > > if available.
> > >
> > > https://github.com/apache/polaris/pull/2914 persists both request id
> and
> > > otel context.
> > >
> > > Do we actually need to persist both?
> > > From the original discussion, the intent of a request ID was to
> correlate
> > > multiple events generated by the same request, for example, linking a
> > > BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we
> > > really need is a single opaque identifier that ties related events
> > > together, regardless of whether it comes from a request ID or an OTel
> > > context.
> > >
> > > If we persist both fields (requestId and otelContext), it can lead to
> > > consumer confusion, since downstream systems would need to decide which
> > one
> > > to rely on, potentially causing inconsistent event correlation. Unless
> > > there are concrete use cases where event consumers need both, I’d
> suggest
> > > persisting a single opaque correlation ID instead. This keeps the
> schema
> > > simpler and ensures consistent event linking.
> > >
> > > Yufei
> > >
> > >
> > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <
> jb@nanthrax.net>
> > > > wrote:
> > > > >
> > > > > Hi
> > > > >
> > > > > Yeah it looks good to me and I agree we have a consensus.
> > > > >
> > > > > Thanks !
> > > > > Regards
> > > > > JB
> > > > >
> > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org>
> a
> > > > écrit :
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Thanks for the good discussion so far. I think we have a broad
> > > > consensus.
> > > > > >
> > > > > > Unless anybody disagrees I am going to move forward with the the
> > > > > > following revised plan, incorporating the latest feedback:
> > > > > >
> > > > > > 1. Restore the original functionality for request IDs, change the
> > > > > > default header name back to x-request-id
> > > > > >
> > > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > > >
> > > > > > 3. Update PolarisEvent, expose request ID if available, expose
> OTel
> > > > > > context if available.
> > > > > >
> > > > > > 4. Update events table SQL schema: insert request ID if
> available,
> > > > > > insert OTel context if available.
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> > jb@nanthrax.net
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi
> > > > > > >
> > > > > > > I think it's better to use w3c_trace_context (better than two
> > > > fields).
> > > > > > >
> > > > > > > Regarding x-request-id, I think it's for a different purpose,
> > > > > > > specifically for persistence. Personally, I think we can
> achieve
> > the
> > > > > > > same with span and events.
> > > > > > > If we want to "distinguish" the two layers, it makes sense to
> > have
> > > > > > > x-request-id (not sure it's super helpful). Else, I think we
> can
> > > > > > > achieve the same with span/event.
> > > > > > >
> > > > > > > Just my $0.01
> > > > > > >
> > > > > > > Regards
> > > > > > > JB
> > > > > > >
> > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as
> > well.
> > > > Great
> > > > > > find, Dmitri!
> > > > > > > >
> > > > > > > > I’m still debating with myself on whether we need to store
> the
> > > > > > `x-request-id` field as part of the Events persistence. Can we
> > think
> > > > of a
> > > > > > good use case where this is more helpful to the user than the
> OTel
> > > > > > Trace/Span IDs? I am making the assumption here that those are
> > still
> > > > being
> > > > > > returned back to the client.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Adnan Hemani
> > > > > > > >
> > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> > adutra@apache.org>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi Dmitri,
> > > > > > > > >
> > > > > > > > > Yes, I think your suggestion to use just one field
> > > > w3c_trace_context
> > > > > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > > > > >
> > > > > > > > > With that, I also think that we are slowly drifting into
> > > > > > > > > implementation considerations; let's get consensus on the
> > general
> > > > > > > > > design first, and we can certainly fine-tune the actual
> Java
> > > > methods
> > > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > > dimas@apache.org> wrote:
> > > > > > > > >>
> > > > > > > > >> Hi All,
> > > > > > > > >>
> > > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > > >>
> > > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > > >>
> > > > > > > > >> Re: (3) Can we abstract request/trace info into a separate
> > > > object
> > > > > > without
> > > > > > > > >> exposing those accessors on the Event class directly? OTel
> > > > defines
> > > > > > > > >> trace/span concepts, but in request ID is a bit foreign to
> > OTel.
> > > > > > Having
> > > > > > > > >> tracing / request ID isolated in java could help with
> > > > maintaining
> > > > > > it and
> > > > > > > > >> potentially supporting other (custom) tracing methods.
> > > > > > > > >>
> > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation data
> > in the
> > > > > > form of a
> > > > > > > > >> standard context propagation string, e.g. W3C
> trace-context
> > [1]
> > > > > > (same value
> > > > > > > > >> as its HTTP header), so the column could be called
> > > > > > w3c_trace_context or
> > > > > > > > >> simply trace_context.
> > > > > > > > >>
> > > > > > > > >> Open question: do we need to write a separate, individual
> > trace
> > > > ID
> > > > > > field in
> > > > > > > > >> SQL? I suppose it is not very useful since correlating it
> to
> > > > other
> > > > > > trace
> > > > > > > > >> data already requires understanding OTel context
> propagation
> > > > and a
> > > > > > query
> > > > > > > > >> against trace_context can still be made using
> > string-matching
> > > > > > clauses. We
> > > > > > > > >> could probably (additionally) store it in the request_id
> > column
> > > > if
> > > > > > the
> > > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > > >>
> > > > > > > > >> As for span ID, I do not really see a use case for
> > persisting it
> > > > > > > > >> individually. It is very specific to OTel trace data
> > > > construction.
> > > > > > > > >>
> > > > > > > > >> Actually, using the W3C trace context [1] encoding
> probably
> > > > makes
> > > > > > sense in
> > > > > > > > >> the java event representation too. Interested callers can
> > easily
> > > > > > decode
> > > > > > > > >> this information since the format is well-defined. As a
> side
> > > > > > benefit, this
> > > > > > > > >> opens opportunities for downstream event consumers to
> > connect
> > > > > > (propagate
> > > > > > > > >> context) to traces that produced events based on the event
> > data
> > > > > > itself,
> > > > > > > > >> without relying on the intermediate frameworks. This may
> be
> > > > > > desirable since
> > > > > > > > >> the current Polaris event persistence impl. writes events
> in
> > > > > > batches, so
> > > > > > > > >> the association to individual requests that produced those
> > > > events
> > > > > > is lost.
> > > > > > > > >> Whether to perform this kind of context propagation will
> be
> > at
> > > > the
> > > > > > > > >> discretion of the event consumer, of course (outside of
> > Polaris
> > > > > > code).
> > > > > > > > >>
> > > > > > > > >> [1]
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > > >>
> > > > > > > > >> WDYT?
> > > > > > > > >>
> > > > > > > > >> Thanks,
> > > > > > > > >> Dmitri.
> > > > > > > > >>
> > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > > adutra@apache.org>
> > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >>> Hi all,
> > > > > > > > >>>
> > > > > > > > >>> Thank you for chiming in; the context around request IDs
> > is now
> > > > > > clear.
> > > > > > > > >>>
> > > > > > > > >>> Trying to summarize this thread into actionable items,
> > here's
> > > > what
> > > > > > I
> > > > > > > > >>> propose:
> > > > > > > > >>>
> > > > > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > > > > >>> - Change the default header name back to x-request-id
> > > > (despite
> > > > > > the
> > > > > > > > >>> x- prefix being deprecated), but keep it configurable as
> > today.
> > > > > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > > > > >>> - Do not generate a request ID if the REST client
> > doesn't
> > > > > > provide one.
> > > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > > >>> - Expose new requestId(), traceId(), spanId() methods,
> > all
> > > > > > nullable.
> > > > > > > > >>> - This would align with the emerging consensus around
> > > > including
> > > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > > >>> - Insert the client-provided request ID into the
> > request_id
> > > > > > > > >>> column; otherwise, insert null.
> > > > > > > > >>> - Add two new nullable columns, trace_id and span_id,
> > and
> > > > > > populate
> > > > > > > > >>> them if OTel is enabled.
> > > > > > > > >>>
> > > > > > > > >>> From our discussions, I think it's important to not
> > conflate
> > > > OTel
> > > > > > > > >>> tracing fields with Envoy tracing fields, which is why I
> > > > suggest we
> > > > > > > > >>> use separate fields / columns for them.
> > > > > > > > >>>
> > > > > > > > >>> Would the above plan work for everyone?
> > > > > > > > >>>
> > > > > > > > >>> Thanks,
> > > > > > > > >>> Alex
> > > > > > > > >>>
> > > > > > > > >>> [1]:
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > > >>>>
> > > > > > > > >>>> Hi all,
> > > > > > > > >>>>
> > > > > > > > >>>> Thanks to Alex for starting this thread - because of
> > this, I’m
> > > > > > just
> > > > > > > > >>> coming to the realization that OTel Trace and Span IDs
> are
> > > > coming
> > > > > > built-in
> > > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > > Request/Correlation ID is
> > > > > > > > >>> likely not needed as a result. My original motivation for
> > > > > > generation of a
> > > > > > > > >>> Request/Correlation ID was to ensure that any client can
> > > > uniquely
> > > > > > identify
> > > > > > > > >>> a request made to Polaris, which would be especially
> > useful for
> > > > > > debugging
> > > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > > >>>>
> > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should
> > remove
> > > > the
> > > > > > > > >>> Request/Correlation ID generation and always use the OTel
> > > > > > trace/span IDs
> > > > > > > > >>> (which come for free with Quarkus) instead for the
> > Correlation
> > > > ID
> > > > > > unless a
> > > > > > > > >>> valid header is present, which would take over as the
> > > > Correlation
> > > > > > ID
> > > > > > > > >>> instead.
> > > > > > > > >>>>
> > > > > > > > >>>> —
> > > > > > > > >>>>
> > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> > intended
> > > > use
> > > > > > case is
> > > > > > > > >>> to provide some sort of correlation between events that
> > have
> > > > > > occurred as
> > > > > > > > >>> part of the same request. For example, if a user makes an
> > > > > > CommitTransaction
> > > > > > > > >>> request, it would be helpful to see all UpdateTable calls
> > that
> > > > > > were made as
> > > > > > > > >>> part of that one user request.
> > > > > > > > >>>>
> > > > > > > > >>>> Best,
> > > > > > > > >>>> Adnan Hemani
> > > > > > > > >>>>
> > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > > > > dimas@apache.org>
> > > > > > > > >>> wrote:
> > > > > > > > >>>>>
> > > > > > > > >>>>> Hi Michael,
> > > > > > > > >>>>>
> > > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > > >>>>>
> > > > > > > > >>>>> Just to confirm: if / when we restore that, Polaris
> will
> > NOT
> > > > > > generate
> > > > > > > > >>> new
> > > > > > > > >>>>> IDs in case the header is not present in the request,
> > > > correct?
> > > > > > > > >>>>>
> > > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > > >>>>>
> > > > > > > > >>>>> What about adding request IDs to events [1][2]? What's
> > the
> > > > > > intended use
> > > > > > > > >>>>> case for that? Could you share some context here too?
> > > > > > > > >>>>>
> > > > > > > > >>>>> Side note: I proposed [2877] flagging event persistence
> > as
> > > > > > "beta" in
> > > > > > > > >>>>> 1.2.0... This discussion adds another point towards
> > that, I
> > > > > > think.
> > > > > > > > >>>>>
> > > > > > > > >>>>> [1]
> > > > > > > > >>>>>
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > > >>>>> [2]
> > > > > > > > >>>>>
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > > >>>>>
> > > > > > > > >>>>> [2877]
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > > >>>>>
> > > > > > > > >>>>> Thanks,
> > > > > > > > >>>>> Dmitri.
> > > > > > > > >>>>>
> > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > >>>>> wrote:
> > > > > > > > >>>>>
> > > > > > > > >>>>>> Hey Dmitri
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> The generating a request id is new code that was added
> > > > after the
> > > > > > > > >>> original
> > > > > > > > >>>>>> x-request-id support. You can see the state from ~1
> year
> > > > ago, we
> > > > > > > > >>> hard-coded
> > > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > > >>>>>> . At some point, it was changed to be configurable,
> > then the
> > > > > > > > >>>>>> ContextResolverFilter filter was refactored/eliminated
> > and
> > > > the
> > > > > > > > >>>>>> RequestIdFilter took its responsibility, but lost some
> > of
> > > > its
> > > > > > original
> > > > > > > > >>>>>> functionality.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> My personal opinion is that restoring support for the
> > > > > > x-request-id
> > > > > > > > >>> header
> > > > > > > > >>>>>> is something that we should do, but if the header
> isn't
> > > > present,
> > > > > > > > >>> falling
> > > > > > > > >>>>>> back on simply using OTel trace ids is good enough
> > (better,
> > > > > > even) than
> > > > > > > > >>>>>> generating another random request id.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Mike
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > > > > > > >>> dimas@apache.org>
> > > > > > > > >>>>>> wrote:
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>> Hi Michael,
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Thanks for the info!
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Working with Envoy's tracing headers makes sense to
> me.
> > > > > > However, I
> > > > > > > > >>>>>> wonder:
> > > > > > > > >>>>>>> why would Polaris need to generate a new request ID
> > inside
> > > > its
> > > > > > > > >>> code?..
> > > > > > > > >>>>>>> and return it in response headers?
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> How important is it to propagate this ID to Polaris
> > Events?
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Thanks,
> > > > > > > > >>>>>>> Dmitri.
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > > > > >>> collado.mike@gmail.com>
> > > > > > > > >>>>>>> wrote:
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>>> I think the original intention for this requestId
> > field
> > > > was to
> > > > > > > > >>> support
> > > > > > > > >>>>>>>> request propagation from load balancers, like Envoy
> (
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the
> > > > default
> > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally a
> > custom
> > > > > > thing,
> > > > > > > > >>> but
> > > > > > > > >>>>>>> then
> > > > > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > > > > Unfortunately,
> > > > > > > > >>>>>> looking
> > > > > > > > >>>>>>>> through the code, I think that the actual functional
> > > > plumbing
> > > > > > has
> > > > > > > > >>> been
> > > > > > > > >>>>>>> lost
> > > > > > > > >>>>>>>> in the course of multiple refactors around the call
> > > > context
> > > > > > and
> > > > > > > > >>>>>>> resolver. I
> > > > > > > > >>>>>>>> don't see references to that property or the
> > underlying
> > > > > > header.
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> Support for the unofficial x-request-id header feels
> > like
> > > > > > something
> > > > > > > > >>> we
> > > > > > > > >>>>>>>> should definitely keep, especially when Polaris is
> one
> > > > > > service in a
> > > > > > > > >>>>>> mesh
> > > > > > > > >>>>>>> of
> > > > > > > > >>>>>>>> services that maybe don't have OTel integration.
> I'm a
> > > > fan of
> > > > > > the
> > > > > > > > >>> OTel
> > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and there
> > are
> > > > many
> > > > > > > > >>>>>> middleware
> > > > > > > > >>>>>>>> layers that know how to forward on the x-request-id
> > > > header.
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> Mike
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > > snazy@snazy.de>
> > > > > > > > >>> wrote:
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with the
> > existing
> > > > > > de-facto
> > > > > > > > >>>>>>>>> standard OTel and make it easy for users to
> integrate
> > > > into
> > > > > > their
> > > > > > > > >>>>>>>>> observability platforms.
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri
> Bourlatchkov <
> > > > > > > > >>>>>> dimas@apache.org>
> > > > > > > > >>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> I certainly support the idea of following OTel
> > > > standards for
> > > > > > > > >>>>>>> achieving
> > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> As to what form the correlation data should take,
> I
> > > > believe
> > > > > > it is
> > > > > > > > >>>>>>>>>> conceptually what the OTel "context" represents.
> > So, I
> > > > > > believe it
> > > > > > > > >>>>>>> makes
> > > > > > > > >>>>>>>>>> sense for Polaris to support standard context
> > > > propagators
> > > > > > at the
> > > > > > > > >>>>>> API
> > > > > > > > >>>>>>>>> layer.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> If the incoming request has OTel context
> > information,
> > > > then
> > > > > > > > >>>>>> returning
> > > > > > > > >>>>>>>> any
> > > > > > > > >>>>>>>>>> other "correlation" data in the response is
> > redundant, I
> > > > > > think.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> If the incoming request does not have OTel context
> > info,
> > > > > > what is
> > > > > > > > >>>>>> the
> > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> > "correlation
> > > > ID"?
> > > > > > How is
> > > > > > > > >>>>>> it
> > > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris
> response
> > > > > > (operation)
> > > > > > > > >>>>>> with
> > > > > > > > >>>>>>>>> events
> > > > > > > > >>>>>>>>>> that might have resulted from its execution, I
> > believe a
> > > > > > more
> > > > > > > > >>>>>> robust
> > > > > > > > >>>>>>>>>> approach would be to propagate the OTel trace info
> > > > > > (starting a new
> > > > > > > > >>>>>>>> trace
> > > > > > > > >>>>>>>>> if
> > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can also
> > > > return
> > > > > > the
> > > > > > > > >>> trace
> > > > > > > > >>>>>>>>> context
> > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit awkward
> > from
> > > > the
> > > > > > OTel
> > > > > > > > >>>>>>>>>> perspective, but might be an option for supporting
> > > > custom
> > > > > > > > >>>>>>> correlators.
> > > > > > > > >>>>>>>> It
> > > > > > > > >>>>>>>>>> could be covered by a feature flag. The header
> name
> > > > could be
> > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Custom correlation code will be able to extract
> the
> > > > trace
> > > > > > ID from
> > > > > > > > >>>>>> the
> > > > > > > > >>>>>>>>>> response and from events and find related data.
> > > > Granted, it
> > > > > > will
> > > > > > > > >>>>>>>> require
> > > > > > > > >>>>>>>>> a
> > > > > > > > >>>>>>>>>> bit more effort for the custom code to decode
> trace
> > IDs
> > > > > > from the
> > > > > > > > >>>>>> OTel
> > > > > > > > >>>>>>>>>> context, but the format is standard and not
> > complex. The
> > > > > > benefit
> > > > > > > > >>>>>> for
> > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate
> > with
> > > > > > > > >>>>>> OTel-compatible
> > > > > > > > >>>>>>>>>> observability platforms regardless of whether any
> > > > particular
> > > > > > > > >>>>>>> deployment
> > > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> WDYT?
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Thanks,
> > > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > > > > > > >>> adutra@apache.org
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID",
> but
> > its
> > > > > > purpose is
> > > > > > > > >>>>>>> not
> > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> > observability
> > > > > > feature to
> > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to
> > rename it
> > > > to
> > > > > > > > >>>>>>>> "correlation
> > > > > > > > >>>>>>>>>>> ID" for better alignment with industry standards
> > [1].
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> However, this PR has brought to light overlaps
> with
> > > > core
> > > > > > > > >>>>>> telemetry
> > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in
> > > > Polaris,
> > > > > > each
> > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID,
> making
> > a
> > > > > > separate
> > > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in
> > > > Polaris
> > > > > > events,
> > > > > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > > > > significantly
> > > > > > > > >>>>>>>> simplify
> > > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span
> > ID as
> > > > the
> > > > > > > > >>>>>>>> correlation
> > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a
> random
> > > > > > correlation
> > > > > > > > >>>>>> ID.
> > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation
> > ID
> > > > > > header is
> > > > > > > > >>>>>>>> present
> > > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is met,
> > generate
> > > > a
> > > > > > random
> > > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best approach
> when a
> > > > > > correlation
> > > > > > > > >>>>>> ID
> > > > > > > > >>>>>>>>>>> header is present in the request. However, I
> > believe it
> > > > > > would be
> > > > > > > > >>>>>>> more
> > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as
> > OTel
> > > > > > offers a
> > > > > > > > >>>>>> more
> > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> > propagation,
> > > > > > e.g. W3C
> > > > > > > > >>>>>>>> Trace
> > > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > > >>>>>>>>>>> Alex
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> [1]:
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > > >>>>>>>>>>> [2]:
> > > > > > > > >>>
> > > > > >
> > > >
> >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>
> > > > > > > > >>>
> > > > > > > >
> > > > > >
> > > >
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi All,
From my POV the use cases for x-request-id and OTel are slightly different.
OTel is an observability use case where cross-linking of various pieces of
data is desired but not required in all cases (data loss is acceptable).
The x-request-id use case appears to have some very specific custom
applications without any obvious relationship to OTel.
Given the current state of this discussion, I think it makes sense to
handle and persist them separately.
However, if we intend x-request-id to act as an observability aid
alternative to OTel, then it might be preferable to have a config flag
indicating which one of them should be added to persistent event data. In
that case we should reuse the same column for them (and possibly add a
"type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...")
WDYT?
Thanks,
Dmitri.
On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <ad...@apache.org> wrote:
> Hi Yufei,
>
> > I’d suggest persisting a single opaque correlation ID instead
>
> The problem I see with this is that readers cannot know the syntax of
> the opaque id, and will have to "probe" using some heuristics, e.g. if
> the id matches the OTel trace context syntax, asume it's OTel,
> otherwise... assume it's something else.
>
> That's not unfeasible, but I wonder if it isn't better to store the 2
> ids separately.
>
> Thanks,
> Alex
>
> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
> >
> > Thanks Alex for raising this discussion. Thanks everyone for chiming in.
> > The plan looks good to me overall.
> >
> > > Update PolarisEvent, expose request ID if available, expose OTel
> context
> > if available.
> >
> > https://github.com/apache/polaris/pull/2914 persists both request id and
> > otel context.
> >
> > Do we actually need to persist both?
> > From the original discussion, the intent of a request ID was to correlate
> > multiple events generated by the same request, for example, linking a
> > BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we
> > really need is a single opaque identifier that ties related events
> > together, regardless of whether it comes from a request ID or an OTel
> > context.
> >
> > If we persist both fields (requestId and otelContext), it can lead to
> > consumer confusion, since downstream systems would need to decide which
> one
> > to rely on, potentially causing inconsistent event correlation. Unless
> > there are concrete use cases where event consumers need both, I’d suggest
> > persisting a single opaque correlation ID instead. This keeps the schema
> > simpler and ensures consistent event linking.
> >
> > Yufei
> >
> >
> > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> > > Hi all,
> > >
> > > Here is the PR: https://github.com/apache/polaris/pull/2914
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> > > wrote:
> > > >
> > > > Hi
> > > >
> > > > Yeah it looks good to me and I agree we have a consensus.
> > > >
> > > > Thanks !
> > > > Regards
> > > > JB
> > > >
> > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org> a
> > > écrit :
> > > >
> > > > > Hi all,
> > > > >
> > > > > Thanks for the good discussion so far. I think we have a broad
> > > consensus.
> > > > >
> > > > > Unless anybody disagrees I am going to move forward with the the
> > > > > following revised plan, incorporating the latest feedback:
> > > > >
> > > > > 1. Restore the original functionality for request IDs, change the
> > > > > default header name back to x-request-id
> > > > >
> > > > > 2. Remove RequestIdGenerator and related functionality.
> > > > >
> > > > > 3. Update PolarisEvent, expose request ID if available, expose OTel
> > > > > context if available.
> > > > >
> > > > > 4. Update events table SQL schema: insert request ID if available,
> > > > > insert OTel context if available.
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <
> jb@nanthrax.net
> > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi
> > > > > >
> > > > > > I think it's better to use w3c_trace_context (better than two
> > > fields).
> > > > > >
> > > > > > Regarding x-request-id, I think it's for a different purpose,
> > > > > > specifically for persistence. Personally, I think we can achieve
> the
> > > > > > same with span and events.
> > > > > > If we want to "distinguish" the two layers, it makes sense to
> have
> > > > > > x-request-id (not sure it's super helpful). Else, I think we can
> > > > > > achieve the same with span/event.
> > > > > >
> > > > > > Just my $0.01
> > > > > >
> > > > > > Regards
> > > > > > JB
> > > > > >
> > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as
> well.
> > > Great
> > > > > find, Dmitri!
> > > > > > >
> > > > > > > I’m still debating with myself on whether we need to store the
> > > > > `x-request-id` field as part of the Events persistence. Can we
> think
> > > of a
> > > > > good use case where this is more helpful to the user than the OTel
> > > > > Trace/Span IDs? I am making the assumption here that those are
> still
> > > being
> > > > > returned back to the client.
> > > > > > >
> > > > > > > Best,
> > > > > > > Adnan Hemani
> > > > > > >
> > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <
> adutra@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > Hi Dmitri,
> > > > > > > >
> > > > > > > > Yes, I think your suggestion to use just one field
> > > w3c_trace_context
> > > > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > > > >
> > > > > > > > With that, I also think that we are slowly drifting into
> > > > > > > > implementation considerations; let's get consensus on the
> general
> > > > > > > > design first, and we can certainly fine-tune the actual Java
> > > methods
> > > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > > dimas@apache.org> wrote:
> > > > > > > >>
> > > > > > > >> Hi All,
> > > > > > > >>
> > > > > > > >> Many thanks for the background info, Adnan!
> > > > > > > >>
> > > > > > > >> +1 to action items proposed by Alex!
> > > > > > > >>
> > > > > > > >> Re: (3) Can we abstract request/trace info into a separate
> > > object
> > > > > without
> > > > > > > >> exposing those accessors on the Event class directly? OTel
> > > defines
> > > > > > > >> trace/span concepts, but in request ID is a bit foreign to
> OTel.
> > > > > Having
> > > > > > > >> tracing / request ID isolated in java could help with
> > > maintaining
> > > > > it and
> > > > > > > >> potentially supporting other (custom) tracing methods.
> > > > > > > >>
> > > > > > > >> Re: (4) I'd like to propose storing OTel correlation data
> in the
> > > > > form of a
> > > > > > > >> standard context propagation string, e.g. W3C trace-context
> [1]
> > > > > (same value
> > > > > > > >> as its HTTP header), so the column could be called
> > > > > w3c_trace_context or
> > > > > > > >> simply trace_context.
> > > > > > > >>
> > > > > > > >> Open question: do we need to write a separate, individual
> trace
> > > ID
> > > > > field in
> > > > > > > >> SQL? I suppose it is not very useful since correlating it to
> > > other
> > > > > trace
> > > > > > > >> data already requires understanding OTel context propagation
> > > and a
> > > > > query
> > > > > > > >> against trace_context can still be made using
> string-matching
> > > > > clauses. We
> > > > > > > >> could probably (additionally) store it in the request_id
> column
> > > if
> > > > > the
> > > > > > > >> Polaris-specific request ID header is not set.
> > > > > > > >>
> > > > > > > >> As for span ID, I do not really see a use case for
> persisting it
> > > > > > > >> individually. It is very specific to OTel trace data
> > > construction.
> > > > > > > >>
> > > > > > > >> Actually, using the W3C trace context [1] encoding probably
> > > makes
> > > > > sense in
> > > > > > > >> the java event representation too. Interested callers can
> easily
> > > > > decode
> > > > > > > >> this information since the format is well-defined. As a side
> > > > > benefit, this
> > > > > > > >> opens opportunities for downstream event consumers to
> connect
> > > > > (propagate
> > > > > > > >> context) to traces that produced events based on the event
> data
> > > > > itself,
> > > > > > > >> without relying on the intermediate frameworks. This may be
> > > > > desirable since
> > > > > > > >> the current Polaris event persistence impl. writes events in
> > > > > batches, so
> > > > > > > >> the association to individual requests that produced those
> > > events
> > > > > is lost.
> > > > > > > >> Whether to perform this kind of context propagation will be
> at
> > > the
> > > > > > > >> discretion of the event consumer, of course (outside of
> Polaris
> > > > > code).
> > > > > > > >>
> > > > > > > >> [1]
> > > > >
> > >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > > >>
> > > > > > > >> WDYT?
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Dmitri.
> > > > > > > >>
> > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > > adutra@apache.org>
> > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hi all,
> > > > > > > >>>
> > > > > > > >>> Thank you for chiming in; the context around request IDs
> is now
> > > > > clear.
> > > > > > > >>>
> > > > > > > >>> Trying to summarize this thread into actionable items,
> here's
> > > what
> > > > > I
> > > > > > > >>> propose:
> > > > > > > >>>
> > > > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > > > >>> - Change the default header name back to x-request-id
> > > (despite
> > > > > the
> > > > > > > >>> x- prefix being deprecated), but keep it configurable as
> today.
> > > > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > > > >>> - Do not generate a request ID if the REST client
> doesn't
> > > > > provide one.
> > > > > > > >>> 3. Update PolarisEvent:
> > > > > > > >>> - Expose new requestId(), traceId(), spanId() methods,
> all
> > > > > nullable.
> > > > > > > >>> - This would align with the emerging consensus around
> > > including
> > > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > > >>> 4. Update events table SQL schema:
> > > > > > > >>> - Insert the client-provided request ID into the
> request_id
> > > > > > > >>> column; otherwise, insert null.
> > > > > > > >>> - Add two new nullable columns, trace_id and span_id,
> and
> > > > > populate
> > > > > > > >>> them if OTel is enabled.
> > > > > > > >>>
> > > > > > > >>> From our discussions, I think it's important to not
> conflate
> > > OTel
> > > > > > > >>> tracing fields with Envoy tracing fields, which is why I
> > > suggest we
> > > > > > > >>> use separate fields / columns for them.
> > > > > > > >>>
> > > > > > > >>> Would the above plan work for everyone?
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Alex
> > > > > > > >>>
> > > > > > > >>> [1]:
> > > > >
> > >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > > >>>>
> > > > > > > >>>> Hi all,
> > > > > > > >>>>
> > > > > > > >>>> Thanks to Alex for starting this thread - because of
> this, I’m
> > > > > just
> > > > > > > >>> coming to the realization that OTel Trace and Span IDs are
> > > coming
> > > > > built-in
> > > > > > > >>> with Quarkus and my previous work to generate a
> > > > > Request/Correlation ID is
> > > > > > > >>> likely not needed as a result. My original motivation for
> > > > > generation of a
> > > > > > > >>> Request/Correlation ID was to ensure that any client can
> > > uniquely
> > > > > identify
> > > > > > > >>> a request made to Polaris, which would be especially
> useful for
> > > > > debugging
> > > > > > > >>> failing requests or identifying call patterns.
> > > > > > > >>>>
> > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should
> remove
> > > the
> > > > > > > >>> Request/Correlation ID generation and always use the OTel
> > > > > trace/span IDs
> > > > > > > >>> (which come for free with Quarkus) instead for the
> Correlation
> > > ID
> > > > > unless a
> > > > > > > >>> valid header is present, which would take over as the
> > > Correlation
> > > > > ID
> > > > > > > >>> instead.
> > > > > > > >>>>
> > > > > > > >>>> —
> > > > > > > >>>>
> > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The
> intended
> > > use
> > > > > case is
> > > > > > > >>> to provide some sort of correlation between events that
> have
> > > > > occurred as
> > > > > > > >>> part of the same request. For example, if a user makes an
> > > > > CommitTransaction
> > > > > > > >>> request, it would be helpful to see all UpdateTable calls
> that
> > > > > were made as
> > > > > > > >>> part of that one user request.
> > > > > > > >>>>
> > > > > > > >>>> Best,
> > > > > > > >>>> Adnan Hemani
> > > > > > > >>>>
> > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > > > dimas@apache.org>
> > > > > > > >>> wrote:
> > > > > > > >>>>>
> > > > > > > >>>>> Hi Michael,
> > > > > > > >>>>>
> > > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > > >>>>>
> > > > > > > >>>>> Just to confirm: if / when we restore that, Polaris will
> NOT
> > > > > generate
> > > > > > > >>> new
> > > > > > > >>>>> IDs in case the header is not present in the request,
> > > correct?
> > > > > > > >>>>>
> > > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > > >>>>>
> > > > > > > >>>>> What about adding request IDs to events [1][2]? What's
> the
> > > > > intended use
> > > > > > > >>>>> case for that? Could you share some context here too?
> > > > > > > >>>>>
> > > > > > > >>>>> Side note: I proposed [2877] flagging event persistence
> as
> > > > > "beta" in
> > > > > > > >>>>> 1.2.0... This discussion adds another point towards
> that, I
> > > > > think.
> > > > > > > >>>>>
> > > > > > > >>>>> [1]
> > > > > > > >>>>>
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > > >>>>> [2]
> > > > > > > >>>>>
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > > >>>>>
> > > > > > > >>>>> [2877]
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > > >>>>>
> > > > > > > >>>>> Thanks,
> > > > > > > >>>>> Dmitri.
> > > > > > > >>>>>
> > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > > >>> collado.mike@gmail.com>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> Hey Dmitri
> > > > > > > >>>>>>
> > > > > > > >>>>>> The generating a request id is new code that was added
> > > after the
> > > > > > > >>> original
> > > > > > > >>>>>> x-request-id support. You can see the state from ~1 year
> > > ago, we
> > > > > > > >>> hard-coded
> > > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > > >>>>>> . At some point, it was changed to be configurable,
> then the
> > > > > > > >>>>>> ContextResolverFilter filter was refactored/eliminated
> and
> > > the
> > > > > > > >>>>>> RequestIdFilter took its responsibility, but lost some
> of
> > > its
> > > > > original
> > > > > > > >>>>>> functionality.
> > > > > > > >>>>>>
> > > > > > > >>>>>> My personal opinion is that restoring support for the
> > > > > x-request-id
> > > > > > > >>> header
> > > > > > > >>>>>> is something that we should do, but if the header isn't
> > > present,
> > > > > > > >>> falling
> > > > > > > >>>>>> back on simply using OTel trace ids is good enough
> (better,
> > > > > even) than
> > > > > > > >>>>>> generating another random request id.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Mike
> > > > > > > >>>>>>
> > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > > > > > >>> dimas@apache.org>
> > > > > > > >>>>>> wrote:
> > > > > > > >>>>>>
> > > > > > > >>>>>>> Hi Michael,
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Thanks for the info!
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Working with Envoy's tracing headers makes sense to me.
> > > > > However, I
> > > > > > > >>>>>> wonder:
> > > > > > > >>>>>>> why would Polaris need to generate a new request ID
> inside
> > > its
> > > > > > > >>> code?..
> > > > > > > >>>>>>> and return it in response headers?
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> How important is it to propagate this ID to Polaris
> Events?
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Thanks,
> > > > > > > >>>>>>> Dmitri.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > > > >>> collado.mike@gmail.com>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>> I think the original intention for this requestId
> field
> > > was to
> > > > > > > >>> support
> > > > > > > >>>>>>>> request propagation from load balancers, like Envoy (
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the
> > > default
> > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally a
> custom
> > > > > thing,
> > > > > > > >>> but
> > > > > > > >>>>>>> then
> > > > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > > > Unfortunately,
> > > > > > > >>>>>> looking
> > > > > > > >>>>>>>> through the code, I think that the actual functional
> > > plumbing
> > > > > has
> > > > > > > >>> been
> > > > > > > >>>>>>> lost
> > > > > > > >>>>>>>> in the course of multiple refactors around the call
> > > context
> > > > > and
> > > > > > > >>>>>>> resolver. I
> > > > > > > >>>>>>>> don't see references to that property or the
> underlying
> > > > > header.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Support for the unofficial x-request-id header feels
> like
> > > > > something
> > > > > > > >>> we
> > > > > > > >>>>>>>> should definitely keep, especially when Polaris is one
> > > > > service in a
> > > > > > > >>>>>> mesh
> > > > > > > >>>>>>> of
> > > > > > > >>>>>>>> services that maybe don't have OTel integration. I'm a
> > > fan of
> > > > > the
> > > > > > > >>> OTel
> > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and there
> are
> > > many
> > > > > > > >>>>>> middleware
> > > > > > > >>>>>>>> layers that know how to forward on the x-request-id
> > > header.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Mike
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > > snazy@snazy.de>
> > > > > > > >>> wrote:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> Yes, we should aim for interoperability with the
> existing
> > > > > de-facto
> > > > > > > >>>>>>>>> standard OTel and make it easy for users to integrate
> > > into
> > > > > their
> > > > > > > >>>>>>>>> observability platforms.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > > > > > >>>>>> dimas@apache.org>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> I certainly support the idea of following OTel
> > > standards for
> > > > > > > >>>>>>> achieving
> > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> As to what form the correlation data should take, I
> > > believe
> > > > > it is
> > > > > > > >>>>>>>>>> conceptually what the OTel "context" represents.
> So, I
> > > > > believe it
> > > > > > > >>>>>>> makes
> > > > > > > >>>>>>>>>> sense for Polaris to support standard context
> > > propagators
> > > > > at the
> > > > > > > >>>>>> API
> > > > > > > >>>>>>>>> layer.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> If the incoming request has OTel context
> information,
> > > then
> > > > > > > >>>>>> returning
> > > > > > > >>>>>>>> any
> > > > > > > >>>>>>>>>> other "correlation" data in the response is
> redundant, I
> > > > > think.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> If the incoming request does not have OTel context
> info,
> > > > > what is
> > > > > > > >>>>>> the
> > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific
> "correlation
> > > ID"?
> > > > > How is
> > > > > > > >>>>>> it
> > > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris response
> > > > > (operation)
> > > > > > > >>>>>> with
> > > > > > > >>>>>>>>> events
> > > > > > > >>>>>>>>>> that might have resulted from its execution, I
> believe a
> > > > > more
> > > > > > > >>>>>> robust
> > > > > > > >>>>>>>>>> approach would be to propagate the OTel trace info
> > > > > (starting a new
> > > > > > > >>>>>>>> trace
> > > > > > > >>>>>>>>> if
> > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can also
> > > return
> > > > > the
> > > > > > > >>> trace
> > > > > > > >>>>>>>>> context
> > > > > > > >>>>>>>>>> in the API response (top span). It's a bit awkward
> from
> > > the
> > > > > OTel
> > > > > > > >>>>>>>>>> perspective, but might be an option for supporting
> > > custom
> > > > > > > >>>>>>> correlators.
> > > > > > > >>>>>>>> It
> > > > > > > >>>>>>>>>> could be covered by a feature flag. The header name
> > > could be
> > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Custom correlation code will be able to extract the
> > > trace
> > > > > ID from
> > > > > > > >>>>>> the
> > > > > > > >>>>>>>>>> response and from events and find related data.
> > > Granted, it
> > > > > will
> > > > > > > >>>>>>>> require
> > > > > > > >>>>>>>>> a
> > > > > > > >>>>>>>>>> bit more effort for the custom code to decode trace
> IDs
> > > > > from the
> > > > > > > >>>>>> OTel
> > > > > > > >>>>>>>>>> context, but the format is standard and not
> complex. The
> > > > > benefit
> > > > > > > >>>>>> for
> > > > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate
> with
> > > > > > > >>>>>> OTel-compatible
> > > > > > > >>>>>>>>>> observability platforms regardless of whether any
> > > particular
> > > > > > > >>>>>>> deployment
> > > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> WDYT?
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Thanks,
> > > > > > > >>>>>>>>>> Dmitri.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > > > > > >>> adutra@apache.org
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>> Hi all,
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but
> its
> > > > > purpose is
> > > > > > > >>>>>>> not
> > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an
> observability
> > > > > feature to
> > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to
> rename it
> > > to
> > > > > > > >>>>>>>> "correlation
> > > > > > > >>>>>>>>>>> ID" for better alignment with industry standards
> [1].
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> However, this PR has brought to light overlaps with
> > > core
> > > > > > > >>>>>> telemetry
> > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in
> > > Polaris,
> > > > > each
> > > > > > > >>>>>>>>>>> request already has a trace ID and span ID, making
> a
> > > > > separate
> > > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in
> > > Polaris
> > > > > events,
> > > > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > > > significantly
> > > > > > > >>>>>>>> simplify
> > > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span
> ID as
> > > the
> > > > > > > >>>>>>>> correlation
> > > > > > > >>>>>>>>>>> ID for the request, instead of generating a random
> > > > > correlation
> > > > > > > >>>>>> ID.
> > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation
> ID
> > > > > header is
> > > > > > > >>>>>>>> present
> > > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is met,
> generate
> > > a
> > > > > random
> > > > > > > >>>>>>>>>>> correlation ID.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> I am somewhat undecided on the best approach when a
> > > > > correlation
> > > > > > > >>>>>> ID
> > > > > > > >>>>>>>>>>> header is present in the request. However, I
> believe it
> > > > > would be
> > > > > > > >>>>>>> more
> > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as
> OTel
> > > > > offers a
> > > > > > > >>>>>> more
> > > > > > > >>>>>>>>>>> robust solution for client-to-server trace
> propagation,
> > > > > e.g. W3C
> > > > > > > >>>>>>>> Trace
> > > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Thanks,
> > > > > > > >>>>>>>>>>> Alex
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> [1]:
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > > >>>>>>>>>>> [2]:
> > > > > > > >>>
> > > > >
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi Yufei,
> I’d suggest persisting a single opaque correlation ID instead
The problem I see with this is that readers cannot know the syntax of
the opaque id, and will have to "probe" using some heuristics, e.g. if
the id matches the OTel trace context syntax, asume it's OTel,
otherwise... assume it's something else.
That's not unfeasible, but I wonder if it isn't better to store the 2
ids separately.
Thanks,
Alex
On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <fl...@gmail.com> wrote:
>
> Thanks Alex for raising this discussion. Thanks everyone for chiming in.
> The plan looks good to me overall.
>
> > Update PolarisEvent, expose request ID if available, expose OTel context
> if available.
>
> https://github.com/apache/polaris/pull/2914 persists both request id and
> otel context.
>
> Do we actually need to persist both?
> From the original discussion, the intent of a request ID was to correlate
> multiple events generated by the same request, for example, linking a
> BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we
> really need is a single opaque identifier that ties related events
> together, regardless of whether it comes from a request ID or an OTel
> context.
>
> If we persist both fields (requestId and otelContext), it can lead to
> consumer confusion, since downstream systems would need to decide which one
> to rely on, potentially causing inconsistent event correlation. Unless
> there are concrete use cases where event consumers need both, I’d suggest
> persisting a single opaque correlation ID instead. This keeps the schema
> simpler and ensures consistent event linking.
>
> Yufei
>
>
> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hi all,
> >
> > Here is the PR: https://github.com/apache/polaris/pull/2914
> >
> > Thanks,
> > Alex
> >
> > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> > >
> > > Hi
> > >
> > > Yeah it looks good to me and I agree we have a consensus.
> > >
> > > Thanks !
> > > Regards
> > > JB
> > >
> > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org> a
> > écrit :
> > >
> > > > Hi all,
> > > >
> > > > Thanks for the good discussion so far. I think we have a broad
> > consensus.
> > > >
> > > > Unless anybody disagrees I am going to move forward with the the
> > > > following revised plan, incorporating the latest feedback:
> > > >
> > > > 1. Restore the original functionality for request IDs, change the
> > > > default header name back to x-request-id
> > > >
> > > > 2. Remove RequestIdGenerator and related functionality.
> > > >
> > > > 3. Update PolarisEvent, expose request ID if available, expose OTel
> > > > context if available.
> > > >
> > > > 4. Update events table SQL schema: insert request ID if available,
> > > > insert OTel context if available.
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > >
> > > > wrote:
> > > > >
> > > > > Hi
> > > > >
> > > > > I think it's better to use w3c_trace_context (better than two
> > fields).
> > > > >
> > > > > Regarding x-request-id, I think it's for a different purpose,
> > > > > specifically for persistence. Personally, I think we can achieve the
> > > > > same with span and events.
> > > > > If we want to "distinguish" the two layers, it makes sense to have
> > > > > x-request-id (not sure it's super helpful). Else, I think we can
> > > > > achieve the same with span/event.
> > > > >
> > > > > Just my $0.01
> > > > >
> > > > > Regards
> > > > > JB
> > > > >
> > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > > <ad...@snowflake.com.invalid> wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as well.
> > Great
> > > > find, Dmitri!
> > > > > >
> > > > > > I’m still debating with myself on whether we need to store the
> > > > `x-request-id` field as part of the Events persistence. Can we think
> > of a
> > > > good use case where this is more helpful to the user than the OTel
> > > > Trace/Span IDs? I am making the assumption here that those are still
> > being
> > > > returned back to the client.
> > > > > >
> > > > > > Best,
> > > > > > Adnan Hemani
> > > > > >
> > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > Hi Dmitri,
> > > > > > >
> > > > > > > Yes, I think your suggestion to use just one field
> > w3c_trace_context
> > > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > > >
> > > > > > > With that, I also think that we are slowly drifting into
> > > > > > > implementation considerations; let's get consensus on the general
> > > > > > > design first, and we can certainly fine-tune the actual Java
> > methods
> > > > > > > and SQL table columns in the future PR. WDYT?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > > dimas@apache.org> wrote:
> > > > > > >>
> > > > > > >> Hi All,
> > > > > > >>
> > > > > > >> Many thanks for the background info, Adnan!
> > > > > > >>
> > > > > > >> +1 to action items proposed by Alex!
> > > > > > >>
> > > > > > >> Re: (3) Can we abstract request/trace info into a separate
> > object
> > > > without
> > > > > > >> exposing those accessors on the Event class directly? OTel
> > defines
> > > > > > >> trace/span concepts, but in request ID is a bit foreign to OTel.
> > > > Having
> > > > > > >> tracing / request ID isolated in java could help with
> > maintaining
> > > > it and
> > > > > > >> potentially supporting other (custom) tracing methods.
> > > > > > >>
> > > > > > >> Re: (4) I'd like to propose storing OTel correlation data in the
> > > > form of a
> > > > > > >> standard context propagation string, e.g. W3C trace-context [1]
> > > > (same value
> > > > > > >> as its HTTP header), so the column could be called
> > > > w3c_trace_context or
> > > > > > >> simply trace_context.
> > > > > > >>
> > > > > > >> Open question: do we need to write a separate, individual trace
> > ID
> > > > field in
> > > > > > >> SQL? I suppose it is not very useful since correlating it to
> > other
> > > > trace
> > > > > > >> data already requires understanding OTel context propagation
> > and a
> > > > query
> > > > > > >> against trace_context can still be made using string-matching
> > > > clauses. We
> > > > > > >> could probably (additionally) store it in the request_id column
> > if
> > > > the
> > > > > > >> Polaris-specific request ID header is not set.
> > > > > > >>
> > > > > > >> As for span ID, I do not really see a use case for persisting it
> > > > > > >> individually. It is very specific to OTel trace data
> > construction.
> > > > > > >>
> > > > > > >> Actually, using the W3C trace context [1] encoding probably
> > makes
> > > > sense in
> > > > > > >> the java event representation too. Interested callers can easily
> > > > decode
> > > > > > >> this information since the format is well-defined. As a side
> > > > benefit, this
> > > > > > >> opens opportunities for downstream event consumers to connect
> > > > (propagate
> > > > > > >> context) to traces that produced events based on the event data
> > > > itself,
> > > > > > >> without relying on the intermediate frameworks. This may be
> > > > desirable since
> > > > > > >> the current Polaris event persistence impl. writes events in
> > > > batches, so
> > > > > > >> the association to individual requests that produced those
> > events
> > > > is lost.
> > > > > > >> Whether to perform this kind of context propagation will be at
> > the
> > > > > > >> discretion of the event consumer, of course (outside of Polaris
> > > > code).
> > > > > > >>
> > > > > > >> [1]
> > > >
> > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > > >>
> > > > > > >> WDYT?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Dmitri.
> > > > > > >>
> > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> > adutra@apache.org>
> > > > wrote:
> > > > > > >>
> > > > > > >>> Hi all,
> > > > > > >>>
> > > > > > >>> Thank you for chiming in; the context around request IDs is now
> > > > clear.
> > > > > > >>>
> > > > > > >>> Trying to summarize this thread into actionable items, here's
> > what
> > > > I
> > > > > > >>> propose:
> > > > > > >>>
> > > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > > >>> - Change the default header name back to x-request-id
> > (despite
> > > > the
> > > > > > >>> x- prefix being deprecated), but keep it configurable as today.
> > > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > > >>> - Do not generate a request ID if the REST client doesn't
> > > > provide one.
> > > > > > >>> 3. Update PolarisEvent:
> > > > > > >>> - Expose new requestId(), traceId(), spanId() methods, all
> > > > nullable.
> > > > > > >>> - This would align with the emerging consensus around
> > including
> > > > > > >>> contextual information in PolarisEvent [1].
> > > > > > >>> 4. Update events table SQL schema:
> > > > > > >>> - Insert the client-provided request ID into the request_id
> > > > > > >>> column; otherwise, insert null.
> > > > > > >>> - Add two new nullable columns, trace_id and span_id, and
> > > > populate
> > > > > > >>> them if OTel is enabled.
> > > > > > >>>
> > > > > > >>> From our discussions, I think it's important to not conflate
> > OTel
> > > > > > >>> tracing fields with Envoy tracing fields, which is why I
> > suggest we
> > > > > > >>> use separate fields / columns for them.
> > > > > > >>>
> > > > > > >>> Would the above plan work for everyone?
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Alex
> > > > > > >>>
> > > > > > >>> [1]:
> > > >
> > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > > >>>>
> > > > > > >>>> Hi all,
> > > > > > >>>>
> > > > > > >>>> Thanks to Alex for starting this thread - because of this, I’m
> > > > just
> > > > > > >>> coming to the realization that OTel Trace and Span IDs are
> > coming
> > > > built-in
> > > > > > >>> with Quarkus and my previous work to generate a
> > > > Request/Correlation ID is
> > > > > > >>> likely not needed as a result. My original motivation for
> > > > generation of a
> > > > > > >>> Request/Correlation ID was to ensure that any client can
> > uniquely
> > > > identify
> > > > > > >>> a request made to Polaris, which would be especially useful for
> > > > debugging
> > > > > > >>> failing requests or identifying call patterns.
> > > > > > >>>>
> > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should remove
> > the
> > > > > > >>> Request/Correlation ID generation and always use the OTel
> > > > trace/span IDs
> > > > > > >>> (which come for free with Quarkus) instead for the Correlation
> > ID
> > > > unless a
> > > > > > >>> valid header is present, which would take over as the
> > Correlation
> > > > ID
> > > > > > >>> instead.
> > > > > > >>>>
> > > > > > >>>> —
> > > > > > >>>>
> > > > > > >>>> To answer Dmitri’s question re Polaris Events: The intended
> > use
> > > > case is
> > > > > > >>> to provide some sort of correlation between events that have
> > > > occurred as
> > > > > > >>> part of the same request. For example, if a user makes an
> > > > CommitTransaction
> > > > > > >>> request, it would be helpful to see all UpdateTable calls that
> > > > were made as
> > > > > > >>> part of that one user request.
> > > > > > >>>>
> > > > > > >>>> Best,
> > > > > > >>>> Adnan Hemani
> > > > > > >>>>
> > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > > dimas@apache.org>
> > > > > > >>> wrote:
> > > > > > >>>>>
> > > > > > >>>>> Hi Michael,
> > > > > > >>>>>
> > > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > > >>>>>
> > > > > > >>>>> Just to confirm: if / when we restore that, Polaris will NOT
> > > > generate
> > > > > > >>> new
> > > > > > >>>>> IDs in case the header is not present in the request,
> > correct?
> > > > > > >>>>>
> > > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > > >>>>>
> > > > > > >>>>> What about adding request IDs to events [1][2]? What's the
> > > > intended use
> > > > > > >>>>> case for that? Could you share some context here too?
> > > > > > >>>>>
> > > > > > >>>>> Side note: I proposed [2877] flagging event persistence as
> > > > "beta" in
> > > > > > >>>>> 1.2.0... This discussion adds another point towards that, I
> > > > think.
> > > > > > >>>>>
> > > > > > >>>>> [1]
> > > > > > >>>>>
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > > >>>>> [2]
> > > > > > >>>>>
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > > >>>>>
> > > > > > >>>>> [2877]
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > > >>>>>
> > > > > > >>>>> Thanks,
> > > > > > >>>>> Dmitri.
> > > > > > >>>>>
> > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > > >>> collado.mike@gmail.com>
> > > > > > >>>>> wrote:
> > > > > > >>>>>
> > > > > > >>>>>> Hey Dmitri
> > > > > > >>>>>>
> > > > > > >>>>>> The generating a request id is new code that was added
> > after the
> > > > > > >>> original
> > > > > > >>>>>> x-request-id support. You can see the state from ~1 year
> > ago, we
> > > > > > >>> hard-coded
> > > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > > >>>>>> . At some point, it was changed to be configurable, then the
> > > > > > >>>>>> ContextResolverFilter filter was refactored/eliminated and
> > the
> > > > > > >>>>>> RequestIdFilter took its responsibility, but lost some of
> > its
> > > > original
> > > > > > >>>>>> functionality.
> > > > > > >>>>>>
> > > > > > >>>>>> My personal opinion is that restoring support for the
> > > > x-request-id
> > > > > > >>> header
> > > > > > >>>>>> is something that we should do, but if the header isn't
> > present,
> > > > > > >>> falling
> > > > > > >>>>>> back on simply using OTel trace ids is good enough (better,
> > > > even) than
> > > > > > >>>>>> generating another random request id.
> > > > > > >>>>>>
> > > > > > >>>>>> Mike
> > > > > > >>>>>>
> > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > > > > >>> dimas@apache.org>
> > > > > > >>>>>> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>>> Hi Michael,
> > > > > > >>>>>>>
> > > > > > >>>>>>> Thanks for the info!
> > > > > > >>>>>>>
> > > > > > >>>>>>> Working with Envoy's tracing headers makes sense to me.
> > > > However, I
> > > > > > >>>>>> wonder:
> > > > > > >>>>>>> why would Polaris need to generate a new request ID inside
> > its
> > > > > > >>> code?..
> > > > > > >>>>>>> and return it in response headers?
> > > > > > >>>>>>>
> > > > > > >>>>>>> How important is it to propagate this ID to Polaris Events?
> > > > > > >>>>>>>
> > > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > > >>>>>>>
> > > > > > >>>>>>> Thanks,
> > > > > > >>>>>>> Dmitri.
> > > > > > >>>>>>>
> > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > > >>> collado.mike@gmail.com>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>>> I think the original intention for this requestId field
> > was to
> > > > > > >>> support
> > > > > > >>>>>>>> request propagation from load balancers, like Envoy (
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the
> > default
> > > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally a custom
> > > > thing,
> > > > > > >>> but
> > > > > > >>>>>>> then
> > > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > > Unfortunately,
> > > > > > >>>>>> looking
> > > > > > >>>>>>>> through the code, I think that the actual functional
> > plumbing
> > > > has
> > > > > > >>> been
> > > > > > >>>>>>> lost
> > > > > > >>>>>>>> in the course of multiple refactors around the call
> > context
> > > > and
> > > > > > >>>>>>> resolver. I
> > > > > > >>>>>>>> don't see references to that property or the underlying
> > > > header.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Support for the unofficial x-request-id header feels like
> > > > something
> > > > > > >>> we
> > > > > > >>>>>>>> should definitely keep, especially when Polaris is one
> > > > service in a
> > > > > > >>>>>> mesh
> > > > > > >>>>>>> of
> > > > > > >>>>>>>> services that maybe don't have OTel integration. I'm a
> > fan of
> > > > the
> > > > > > >>> OTel
> > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and there are
> > many
> > > > > > >>>>>> middleware
> > > > > > >>>>>>>> layers that know how to forward on the x-request-id
> > header.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Mike
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> > snazy@snazy.de>
> > > > > > >>> wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>> Yes, we should aim for interoperability with the existing
> > > > de-facto
> > > > > > >>>>>>>>> standard OTel and make it easy for users to integrate
> > into
> > > > their
> > > > > > >>>>>>>>> observability platforms.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > > > > >>>>>> dimas@apache.org>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Hi Alex and All,
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> I certainly support the idea of following OTel
> > standards for
> > > > > > >>>>>>> achieving
> > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> As to what form the correlation data should take, I
> > believe
> > > > it is
> > > > > > >>>>>>>>>> conceptually what the OTel "context" represents. So, I
> > > > believe it
> > > > > > >>>>>>> makes
> > > > > > >>>>>>>>>> sense for Polaris to support standard context
> > propagators
> > > > at the
> > > > > > >>>>>> API
> > > > > > >>>>>>>>> layer.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> If the incoming request has OTel context information,
> > then
> > > > > > >>>>>> returning
> > > > > > >>>>>>>> any
> > > > > > >>>>>>>>>> other "correlation" data in the response is redundant, I
> > > > think.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> If the incoming request does not have OTel context info,
> > > > what is
> > > > > > >>>>>> the
> > > > > > >>>>>>>>>> purpose of generating a Polaris-specific "correlation
> > ID"?
> > > > How is
> > > > > > >>>>>> it
> > > > > > >>>>>>>>>> envisioned to be used?
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> If the intention is to correlate a Polaris response
> > > > (operation)
> > > > > > >>>>>> with
> > > > > > >>>>>>>>> events
> > > > > > >>>>>>>>>> that might have resulted from its execution, I believe a
> > > > more
> > > > > > >>>>>> robust
> > > > > > >>>>>>>>>> approach would be to propagate the OTel trace info
> > > > (starting a new
> > > > > > >>>>>>>> trace
> > > > > > >>>>>>>>> if
> > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can also
> > return
> > > > the
> > > > > > >>> trace
> > > > > > >>>>>>>>> context
> > > > > > >>>>>>>>>> in the API response (top span). It's a bit awkward from
> > the
> > > > OTel
> > > > > > >>>>>>>>>> perspective, but might be an option for supporting
> > custom
> > > > > > >>>>>>> correlators.
> > > > > > >>>>>>>> It
> > > > > > >>>>>>>>>> could be covered by a feature flag. The header name
> > could be
> > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Custom correlation code will be able to extract the
> > trace
> > > > ID from
> > > > > > >>>>>> the
> > > > > > >>>>>>>>>> response and from events and find related data.
> > Granted, it
> > > > will
> > > > > > >>>>>>>> require
> > > > > > >>>>>>>>> a
> > > > > > >>>>>>>>>> bit more effort for the custom code to decode trace IDs
> > > > from the
> > > > > > >>>>>> OTel
> > > > > > >>>>>>>>>> context, but the format is standard and not complex. The
> > > > benefit
> > > > > > >>>>>> for
> > > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate with
> > > > > > >>>>>> OTel-compatible
> > > > > > >>>>>>>>>> observability platforms regardless of whether any
> > particular
> > > > > > >>>>>>> deployment
> > > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> WDYT?
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Thanks,
> > > > > > >>>>>>>>>> Dmitri.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > > > > >>> adutra@apache.org
> > > > > > >>>>>>>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>> Hi all,
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its
> > > > purpose is
> > > > > > >>>>>>> not
> > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an observability
> > > > feature to
> > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it
> > to
> > > > > > >>>>>>>> "correlation
> > > > > > >>>>>>>>>>> ID" for better alignment with industry standards [1].
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> However, this PR has brought to light overlaps with
> > core
> > > > > > >>>>>> telemetry
> > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in
> > Polaris,
> > > > each
> > > > > > >>>>>>>>>>> request already has a trace ID and span ID, making a
> > > > separate
> > > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in
> > Polaris
> > > > events,
> > > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > > significantly
> > > > > > >>>>>>>> simplify
> > > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as
> > the
> > > > > > >>>>>>>> correlation
> > > > > > >>>>>>>>>>> ID for the request, instead of generating a random
> > > > correlation
> > > > > > >>>>>> ID.
> > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID
> > > > header is
> > > > > > >>>>>>>> present
> > > > > > >>>>>>>>>>> in the request, use it.
> > > > > > >>>>>>>>>>> 3. If neither of the above conditions is met, generate
> > a
> > > > random
> > > > > > >>>>>>>>>>> correlation ID.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> I am somewhat undecided on the best approach when a
> > > > correlation
> > > > > > >>>>>> ID
> > > > > > >>>>>>>>>>> header is present in the request. However, I believe it
> > > > would be
> > > > > > >>>>>>> more
> > > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel
> > > > offers a
> > > > > > >>>>>> more
> > > > > > >>>>>>>>>>> robust solution for client-to-server trace propagation,
> > > > e.g. W3C
> > > > > > >>>>>>>> Trace
> > > > > > >>>>>>>>>>> Context propagation [2].
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Thanks,
> > > > > > >>>>>>>>>>> Alex
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> [1]:
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > > >>>>>>>>>>> [2]:
> > > > > > >>>
> > > >
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>
> > > > > > >>>
> > > > > >
> > > >
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Yufei Gu <fl...@gmail.com>.
Thanks Alex for raising this discussion. Thanks everyone for chiming in.
The plan looks good to me overall.
> Update PolarisEvent, expose request ID if available, expose OTel context
if available.
https://github.com/apache/polaris/pull/2914 persists both request id and
otel context.
Do we actually need to persist both?
From the original discussion, the intent of a request ID was to correlate
multiple events generated by the same request, for example, linking a
BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we
really need is a single opaque identifier that ties related events
together, regardless of whether it comes from a request ID or an OTel
context.
If we persist both fields (requestId and otelContext), it can lead to
consumer confusion, since downstream systems would need to decide which one
to rely on, potentially causing inconsistent event correlation. Unless
there are concrete use cases where event consumers need both, I’d suggest
persisting a single opaque correlation ID instead. This keeps the schema
simpler and ensures consistent event linking.
Yufei
On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <ad...@apache.org> wrote:
> Hi all,
>
> Here is the PR: https://github.com/apache/polaris/pull/2914
>
> Thanks,
> Alex
>
> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >
> > Hi
> >
> > Yeah it looks good to me and I agree we have a consensus.
> >
> > Thanks !
> > Regards
> > JB
> >
> > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org> a
> écrit :
> >
> > > Hi all,
> > >
> > > Thanks for the good discussion so far. I think we have a broad
> consensus.
> > >
> > > Unless anybody disagrees I am going to move forward with the the
> > > following revised plan, incorporating the latest feedback:
> > >
> > > 1. Restore the original functionality for request IDs, change the
> > > default header name back to x-request-id
> > >
> > > 2. Remove RequestIdGenerator and related functionality.
> > >
> > > 3. Update PolarisEvent, expose request ID if available, expose OTel
> > > context if available.
> > >
> > > 4. Update events table SQL schema: insert request ID if available,
> > > insert OTel context if available.
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <jb@nanthrax.net
> >
> > > wrote:
> > > >
> > > > Hi
> > > >
> > > > I think it's better to use w3c_trace_context (better than two
> fields).
> > > >
> > > > Regarding x-request-id, I think it's for a different purpose,
> > > > specifically for persistence. Personally, I think we can achieve the
> > > > same with span and events.
> > > > If we want to "distinguish" the two layers, it makes sense to have
> > > > x-request-id (not sure it's super helpful). Else, I think we can
> > > > achieve the same with span/event.
> > > >
> > > > Just my $0.01
> > > >
> > > > Regards
> > > > JB
> > > >
> > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > > <ad...@snowflake.com.invalid> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as well.
> Great
> > > find, Dmitri!
> > > > >
> > > > > I’m still debating with myself on whether we need to store the
> > > `x-request-id` field as part of the Events persistence. Can we think
> of a
> > > good use case where this is more helpful to the user than the OTel
> > > Trace/Span IDs? I am making the assumption here that those are still
> being
> > > returned back to the client.
> > > > >
> > > > > Best,
> > > > > Adnan Hemani
> > > > >
> > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org>
> > > wrote:
> > > > > >
> > > > > > Hi Dmitri,
> > > > > >
> > > > > > Yes, I think your suggestion to use just one field
> w3c_trace_context
> > > > > > makes more sense than two fields (span_id and trace_id).
> > > > > >
> > > > > > With that, I also think that we are slowly drifting into
> > > > > > implementation considerations; let's get consensus on the general
> > > > > > design first, and we can certainly fine-tune the actual Java
> methods
> > > > > > and SQL table columns in the future PR. WDYT?
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > > dimas@apache.org> wrote:
> > > > > >>
> > > > > >> Hi All,
> > > > > >>
> > > > > >> Many thanks for the background info, Adnan!
> > > > > >>
> > > > > >> +1 to action items proposed by Alex!
> > > > > >>
> > > > > >> Re: (3) Can we abstract request/trace info into a separate
> object
> > > without
> > > > > >> exposing those accessors on the Event class directly? OTel
> defines
> > > > > >> trace/span concepts, but in request ID is a bit foreign to OTel.
> > > Having
> > > > > >> tracing / request ID isolated in java could help with
> maintaining
> > > it and
> > > > > >> potentially supporting other (custom) tracing methods.
> > > > > >>
> > > > > >> Re: (4) I'd like to propose storing OTel correlation data in the
> > > form of a
> > > > > >> standard context propagation string, e.g. W3C trace-context [1]
> > > (same value
> > > > > >> as its HTTP header), so the column could be called
> > > w3c_trace_context or
> > > > > >> simply trace_context.
> > > > > >>
> > > > > >> Open question: do we need to write a separate, individual trace
> ID
> > > field in
> > > > > >> SQL? I suppose it is not very useful since correlating it to
> other
> > > trace
> > > > > >> data already requires understanding OTel context propagation
> and a
> > > query
> > > > > >> against trace_context can still be made using string-matching
> > > clauses. We
> > > > > >> could probably (additionally) store it in the request_id column
> if
> > > the
> > > > > >> Polaris-specific request ID header is not set.
> > > > > >>
> > > > > >> As for span ID, I do not really see a use case for persisting it
> > > > > >> individually. It is very specific to OTel trace data
> construction.
> > > > > >>
> > > > > >> Actually, using the W3C trace context [1] encoding probably
> makes
> > > sense in
> > > > > >> the java event representation too. Interested callers can easily
> > > decode
> > > > > >> this information since the format is well-defined. As a side
> > > benefit, this
> > > > > >> opens opportunities for downstream event consumers to connect
> > > (propagate
> > > > > >> context) to traces that produced events based on the event data
> > > itself,
> > > > > >> without relying on the intermediate frameworks. This may be
> > > desirable since
> > > > > >> the current Polaris event persistence impl. writes events in
> > > batches, so
> > > > > >> the association to individual requests that produced those
> events
> > > is lost.
> > > > > >> Whether to perform this kind of context propagation will be at
> the
> > > > > >> discretion of the event consumer, of course (outside of Polaris
> > > code).
> > > > > >>
> > > > > >> [1]
> > >
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > > >>
> > > > > >> WDYT?
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Dmitri.
> > > > > >>
> > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <
> adutra@apache.org>
> > > wrote:
> > > > > >>
> > > > > >>> Hi all,
> > > > > >>>
> > > > > >>> Thank you for chiming in; the context around request IDs is now
> > > clear.
> > > > > >>>
> > > > > >>> Trying to summarize this thread into actionable items, here's
> what
> > > I
> > > > > >>> propose:
> > > > > >>>
> > > > > >>> 1. Restore the original functionality for request IDs.
> > > > > >>> - Change the default header name back to x-request-id
> (despite
> > > the
> > > > > >>> x- prefix being deprecated), but keep it configurable as today.
> > > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > > >>> - Do not generate a request ID if the REST client doesn't
> > > provide one.
> > > > > >>> 3. Update PolarisEvent:
> > > > > >>> - Expose new requestId(), traceId(), spanId() methods, all
> > > nullable.
> > > > > >>> - This would align with the emerging consensus around
> including
> > > > > >>> contextual information in PolarisEvent [1].
> > > > > >>> 4. Update events table SQL schema:
> > > > > >>> - Insert the client-provided request ID into the request_id
> > > > > >>> column; otherwise, insert null.
> > > > > >>> - Add two new nullable columns, trace_id and span_id, and
> > > populate
> > > > > >>> them if OTel is enabled.
> > > > > >>>
> > > > > >>> From our discussions, I think it's important to not conflate
> OTel
> > > > > >>> tracing fields with Envoy tracing fields, which is why I
> suggest we
> > > > > >>> use separate fields / columns for them.
> > > > > >>>
> > > > > >>> Would the above plan work for everyone?
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Alex
> > > > > >>>
> > > > > >>> [1]:
> > >
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > > >>>
> > > > > >>>
> > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > > >>>>
> > > > > >>>> Hi all,
> > > > > >>>>
> > > > > >>>> Thanks to Alex for starting this thread - because of this, I’m
> > > just
> > > > > >>> coming to the realization that OTel Trace and Span IDs are
> coming
> > > built-in
> > > > > >>> with Quarkus and my previous work to generate a
> > > Request/Correlation ID is
> > > > > >>> likely not needed as a result. My original motivation for
> > > generation of a
> > > > > >>> Request/Correlation ID was to ensure that any client can
> uniquely
> > > identify
> > > > > >>> a request made to Polaris, which would be especially useful for
> > > debugging
> > > > > >>> failing requests or identifying call patterns.
> > > > > >>>>
> > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should remove
> the
> > > > > >>> Request/Correlation ID generation and always use the OTel
> > > trace/span IDs
> > > > > >>> (which come for free with Quarkus) instead for the Correlation
> ID
> > > unless a
> > > > > >>> valid header is present, which would take over as the
> Correlation
> > > ID
> > > > > >>> instead.
> > > > > >>>>
> > > > > >>>> —
> > > > > >>>>
> > > > > >>>> To answer Dmitri’s question re Polaris Events: The intended
> use
> > > case is
> > > > > >>> to provide some sort of correlation between events that have
> > > occurred as
> > > > > >>> part of the same request. For example, if a user makes an
> > > CommitTransaction
> > > > > >>> request, it would be helpful to see all UpdateTable calls that
> > > were made as
> > > > > >>> part of that one user request.
> > > > > >>>>
> > > > > >>>> Best,
> > > > > >>>> Adnan Hemani
> > > > > >>>>
> > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > > dimas@apache.org>
> > > > > >>> wrote:
> > > > > >>>>>
> > > > > >>>>> Hi Michael,
> > > > > >>>>>
> > > > > >>>>> Logging x-request-id headers makes sense.
> > > > > >>>>>
> > > > > >>>>> Just to confirm: if / when we restore that, Polaris will NOT
> > > generate
> > > > > >>> new
> > > > > >>>>> IDs in case the header is not present in the request,
> correct?
> > > > > >>>>>
> > > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > > >>>>>
> > > > > >>>>> What about adding request IDs to events [1][2]? What's the
> > > intended use
> > > > > >>>>> case for that? Could you share some context here too?
> > > > > >>>>>
> > > > > >>>>> Side note: I proposed [2877] flagging event persistence as
> > > "beta" in
> > > > > >>>>> 1.2.0... This discussion adds another point towards that, I
> > > think.
> > > > > >>>>>
> > > > > >>>>> [1]
> > > > > >>>>>
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > > >>>>> [2]
> > > > > >>>>>
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > > >>>>>
> > > > > >>>>> [2877]
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > > >>>>>
> > > > > >>>>> Thanks,
> > > > > >>>>> Dmitri.
> > > > > >>>>>
> > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > > >>> collado.mike@gmail.com>
> > > > > >>>>> wrote:
> > > > > >>>>>
> > > > > >>>>>> Hey Dmitri
> > > > > >>>>>>
> > > > > >>>>>> The generating a request id is new code that was added
> after the
> > > > > >>> original
> > > > > >>>>>> x-request-id support. You can see the state from ~1 year
> ago, we
> > > > > >>> hard-coded
> > > > > >>>>>> request_id as the header we used for the MDC -
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > > >>>>>> . At some point, it was changed to be configurable, then the
> > > > > >>>>>> ContextResolverFilter filter was refactored/eliminated and
> the
> > > > > >>>>>> RequestIdFilter took its responsibility, but lost some of
> its
> > > original
> > > > > >>>>>> functionality.
> > > > > >>>>>>
> > > > > >>>>>> My personal opinion is that restoring support for the
> > > x-request-id
> > > > > >>> header
> > > > > >>>>>> is something that we should do, but if the header isn't
> present,
> > > > > >>> falling
> > > > > >>>>>> back on simply using OTel trace ids is good enough (better,
> > > even) than
> > > > > >>>>>> generating another random request id.
> > > > > >>>>>>
> > > > > >>>>>> Mike
> > > > > >>>>>>
> > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > > > >>> dimas@apache.org>
> > > > > >>>>>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> Hi Michael,
> > > > > >>>>>>>
> > > > > >>>>>>> Thanks for the info!
> > > > > >>>>>>>
> > > > > >>>>>>> Working with Envoy's tracing headers makes sense to me.
> > > However, I
> > > > > >>>>>> wonder:
> > > > > >>>>>>> why would Polaris need to generate a new request ID inside
> its
> > > > > >>> code?..
> > > > > >>>>>>> and return it in response headers?
> > > > > >>>>>>>
> > > > > >>>>>>> How important is it to propagate this ID to Polaris Events?
> > > > > >>>>>>>
> > > > > >>>>>>> I'm just trying to understand the full context :)
> > > > > >>>>>>>
> > > > > >>>>>>> Thanks,
> > > > > >>>>>>> Dmitri.
> > > > > >>>>>>>
> > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > > >>> collado.mike@gmail.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> I think the original intention for this requestId field
> was to
> > > > > >>> support
> > > > > >>>>>>>> request propagation from load balancers, like Envoy (
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the
> default
> > > > > >>>>>>>> is Polaris-Request-Id - I think it was originally a custom
> > > thing,
> > > > > >>> but
> > > > > >>>>>>> then
> > > > > >>>>>>>> we changed to integrate with existing conventions.
> > > Unfortunately,
> > > > > >>>>>> looking
> > > > > >>>>>>>> through the code, I think that the actual functional
> plumbing
> > > has
> > > > > >>> been
> > > > > >>>>>>> lost
> > > > > >>>>>>>> in the course of multiple refactors around the call
> context
> > > and
> > > > > >>>>>>> resolver. I
> > > > > >>>>>>>> don't see references to that property or the underlying
> > > header.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Support for the unofficial x-request-id header feels like
> > > something
> > > > > >>> we
> > > > > >>>>>>>> should definitely keep, especially when Polaris is one
> > > service in a
> > > > > >>>>>> mesh
> > > > > >>>>>>> of
> > > > > >>>>>>>> services that maybe don't have OTel integration. I'm a
> fan of
> > > the
> > > > > >>> OTel
> > > > > >>>>>>>> standard, but it's not entirely ubiquitous and there are
> many
> > > > > >>>>>> middleware
> > > > > >>>>>>>> layers that know how to forward on the x-request-id
> header.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Mike
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <
> snazy@snazy.de>
> > > > > >>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> Yes, we should aim for interoperability with the existing
> > > de-facto
> > > > > >>>>>>>>> standard OTel and make it easy for users to integrate
> into
> > > their
> > > > > >>>>>>>>> observability platforms.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > > > >>>>>> dimas@apache.org>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Hi Alex and All,
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> I certainly support the idea of following OTel
> standards for
> > > > > >>>>>>> achieving
> > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> As to what form the correlation data should take, I
> believe
> > > it is
> > > > > >>>>>>>>>> conceptually what the OTel "context" represents. So, I
> > > believe it
> > > > > >>>>>>> makes
> > > > > >>>>>>>>>> sense for Polaris to support standard context
> propagators
> > > at the
> > > > > >>>>>> API
> > > > > >>>>>>>>> layer.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> If the incoming request has OTel context information,
> then
> > > > > >>>>>> returning
> > > > > >>>>>>>> any
> > > > > >>>>>>>>>> other "correlation" data in the response is redundant, I
> > > think.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> If the incoming request does not have OTel context info,
> > > what is
> > > > > >>>>>> the
> > > > > >>>>>>>>>> purpose of generating a Polaris-specific "correlation
> ID"?
> > > How is
> > > > > >>>>>> it
> > > > > >>>>>>>>>> envisioned to be used?
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> If the intention is to correlate a Polaris response
> > > (operation)
> > > > > >>>>>> with
> > > > > >>>>>>>>> events
> > > > > >>>>>>>>>> that might have resulted from its execution, I believe a
> > > more
> > > > > >>>>>> robust
> > > > > >>>>>>>>>> approach would be to propagate the OTel trace info
> > > (starting a new
> > > > > >>>>>>>> trace
> > > > > >>>>>>>>> if
> > > > > >>>>>>>>>> necessary) into event data. Then, Polaris can also
> return
> > > the
> > > > > >>> trace
> > > > > >>>>>>>>> context
> > > > > >>>>>>>>>> in the API response (top span). It's a bit awkward from
> the
> > > OTel
> > > > > >>>>>>>>>> perspective, but might be an option for supporting
> custom
> > > > > >>>>>>> correlators.
> > > > > >>>>>>>> It
> > > > > >>>>>>>>>> could be covered by a feature flag. The header name
> could be
> > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Custom correlation code will be able to extract the
> trace
> > > ID from
> > > > > >>>>>> the
> > > > > >>>>>>>>>> response and from events and find related data.
> Granted, it
> > > will
> > > > > >>>>>>>> require
> > > > > >>>>>>>>> a
> > > > > >>>>>>>>>> bit more effort for the custom code to decode trace IDs
> > > from the
> > > > > >>>>>> OTel
> > > > > >>>>>>>>>> context, but the format is standard and not complex. The
> > > benefit
> > > > > >>>>>> for
> > > > > >>>>>>>>>> Polaris, though, is that it can easily integrate with
> > > > > >>>>>> OTel-compatible
> > > > > >>>>>>>>>> observability platforms regardless of whether any
> particular
> > > > > >>>>>>> deployment
> > > > > >>>>>>>>>> uses custom correlators or not.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> WDYT?
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Thanks,
> > > > > >>>>>>>>>> Dmitri.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > > > >>> adutra@apache.org
> > > > > >>>>>>>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> Hi all,
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its
> > > purpose is
> > > > > >>>>>>> not
> > > > > >>>>>>>>>>> entirely clear. It seems to serve as an observability
> > > feature to
> > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it
> to
> > > > > >>>>>>>> "correlation
> > > > > >>>>>>>>>>> ID" for better alignment with industry standards [1].
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> However, this PR has brought to light overlaps with
> core
> > > > > >>>>>> telemetry
> > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in
> Polaris,
> > > each
> > > > > >>>>>>>>>>> request already has a trace ID and span ID, making a
> > > separate
> > > > > >>>>>>>>>>> correlation ID redundant.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in
> Polaris
> > > events,
> > > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > > significantly
> > > > > >>>>>>>> simplify
> > > > > >>>>>>>>>>> correlation of events with other traces.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as
> the
> > > > > >>>>>>>> correlation
> > > > > >>>>>>>>>>> ID for the request, instead of generating a random
> > > correlation
> > > > > >>>>>> ID.
> > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID
> > > header is
> > > > > >>>>>>>> present
> > > > > >>>>>>>>>>> in the request, use it.
> > > > > >>>>>>>>>>> 3. If neither of the above conditions is met, generate
> a
> > > random
> > > > > >>>>>>>>>>> correlation ID.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> I am somewhat undecided on the best approach when a
> > > correlation
> > > > > >>>>>> ID
> > > > > >>>>>>>>>>> header is present in the request. However, I believe it
> > > would be
> > > > > >>>>>>> more
> > > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel
> > > offers a
> > > > > >>>>>> more
> > > > > >>>>>>>>>>> robust solution for client-to-server trace propagation,
> > > e.g. W3C
> > > > > >>>>>>>> Trace
> > > > > >>>>>>>>>>> Context propagation [2].
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Please share your thoughts!
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>> Alex
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> [1]:
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > > >>>>>>>>>>> [2]:
> > > > > >>>
> > >
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>
> > > > > >>>
> > > > >
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,
Here is the PR: https://github.com/apache/polaris/pull/2914
Thanks,
Alex
On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi
>
> Yeah it looks good to me and I agree we have a consensus.
>
> Thanks !
> Regards
> JB
>
> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org> a écrit :
>
> > Hi all,
> >
> > Thanks for the good discussion so far. I think we have a broad consensus.
> >
> > Unless anybody disagrees I am going to move forward with the the
> > following revised plan, incorporating the latest feedback:
> >
> > 1. Restore the original functionality for request IDs, change the
> > default header name back to x-request-id
> >
> > 2. Remove RequestIdGenerator and related functionality.
> >
> > 3. Update PolarisEvent, expose request ID if available, expose OTel
> > context if available.
> >
> > 4. Update events table SQL schema: insert request ID if available,
> > insert OTel context if available.
> >
> > Thanks,
> > Alex
> >
> > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> > >
> > > Hi
> > >
> > > I think it's better to use w3c_trace_context (better than two fields).
> > >
> > > Regarding x-request-id, I think it's for a different purpose,
> > > specifically for persistence. Personally, I think we can achieve the
> > > same with span and events.
> > > If we want to "distinguish" the two layers, it makes sense to have
> > > x-request-id (not sure it's super helpful). Else, I think we can
> > > achieve the same with span/event.
> > >
> > > Just my $0.01
> > >
> > > Regards
> > > JB
> > >
> > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > > <ad...@snowflake.com.invalid> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as well. Great
> > find, Dmitri!
> > > >
> > > > I’m still debating with myself on whether we need to store the
> > `x-request-id` field as part of the Events persistence. Can we think of a
> > good use case where this is more helpful to the user than the OTel
> > Trace/Span IDs? I am making the assumption here that those are still being
> > returned back to the client.
> > > >
> > > > Best,
> > > > Adnan Hemani
> > > >
> > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org>
> > wrote:
> > > > >
> > > > > Hi Dmitri,
> > > > >
> > > > > Yes, I think your suggestion to use just one field w3c_trace_context
> > > > > makes more sense than two fields (span_id and trace_id).
> > > > >
> > > > > With that, I also think that we are slowly drifting into
> > > > > implementation considerations; let's get consensus on the general
> > > > > design first, and we can certainly fine-tune the actual Java methods
> > > > > and SQL table columns in the future PR. WDYT?
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> > dimas@apache.org> wrote:
> > > > >>
> > > > >> Hi All,
> > > > >>
> > > > >> Many thanks for the background info, Adnan!
> > > > >>
> > > > >> +1 to action items proposed by Alex!
> > > > >>
> > > > >> Re: (3) Can we abstract request/trace info into a separate object
> > without
> > > > >> exposing those accessors on the Event class directly? OTel defines
> > > > >> trace/span concepts, but in request ID is a bit foreign to OTel.
> > Having
> > > > >> tracing / request ID isolated in java could help with maintaining
> > it and
> > > > >> potentially supporting other (custom) tracing methods.
> > > > >>
> > > > >> Re: (4) I'd like to propose storing OTel correlation data in the
> > form of a
> > > > >> standard context propagation string, e.g. W3C trace-context [1]
> > (same value
> > > > >> as its HTTP header), so the column could be called
> > w3c_trace_context or
> > > > >> simply trace_context.
> > > > >>
> > > > >> Open question: do we need to write a separate, individual trace ID
> > field in
> > > > >> SQL? I suppose it is not very useful since correlating it to other
> > trace
> > > > >> data already requires understanding OTel context propagation and a
> > query
> > > > >> against trace_context can still be made using string-matching
> > clauses. We
> > > > >> could probably (additionally) store it in the request_id column if
> > the
> > > > >> Polaris-specific request ID header is not set.
> > > > >>
> > > > >> As for span ID, I do not really see a use case for persisting it
> > > > >> individually. It is very specific to OTel trace data construction.
> > > > >>
> > > > >> Actually, using the W3C trace context [1] encoding probably makes
> > sense in
> > > > >> the java event representation too. Interested callers can easily
> > decode
> > > > >> this information since the format is well-defined. As a side
> > benefit, this
> > > > >> opens opportunities for downstream event consumers to connect
> > (propagate
> > > > >> context) to traces that produced events based on the event data
> > itself,
> > > > >> without relying on the intermediate frameworks. This may be
> > desirable since
> > > > >> the current Polaris event persistence impl. writes events in
> > batches, so
> > > > >> the association to individual requests that produced those events
> > is lost.
> > > > >> Whether to perform this kind of context propagation will be at the
> > > > >> discretion of the event consumer, of course (outside of Polaris
> > code).
> > > > >>
> > > > >> [1]
> > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > > >>
> > > > >> WDYT?
> > > > >>
> > > > >> Thanks,
> > > > >> Dmitri.
> > > > >>
> > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > > > >>
> > > > >>> Hi all,
> > > > >>>
> > > > >>> Thank you for chiming in; the context around request IDs is now
> > clear.
> > > > >>>
> > > > >>> Trying to summarize this thread into actionable items, here's what
> > I
> > > > >>> propose:
> > > > >>>
> > > > >>> 1. Restore the original functionality for request IDs.
> > > > >>> - Change the default header name back to x-request-id (despite
> > the
> > > > >>> x- prefix being deprecated), but keep it configurable as today.
> > > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > > >>> - Do not generate a request ID if the REST client doesn't
> > provide one.
> > > > >>> 3. Update PolarisEvent:
> > > > >>> - Expose new requestId(), traceId(), spanId() methods, all
> > nullable.
> > > > >>> - This would align with the emerging consensus around including
> > > > >>> contextual information in PolarisEvent [1].
> > > > >>> 4. Update events table SQL schema:
> > > > >>> - Insert the client-provided request ID into the request_id
> > > > >>> column; otherwise, insert null.
> > > > >>> - Add two new nullable columns, trace_id and span_id, and
> > populate
> > > > >>> them if OTel is enabled.
> > > > >>>
> > > > >>> From our discussions, I think it's important to not conflate OTel
> > > > >>> tracing fields with Envoy tracing fields, which is why I suggest we
> > > > >>> use separate fields / columns for them.
> > > > >>>
> > > > >>> Would the above plan work for everyone?
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Alex
> > > > >>>
> > > > >>> [1]:
> > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > > >>>
> > > > >>>
> > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > > >>> <ad...@snowflake.com.invalid> wrote:
> > > > >>>>
> > > > >>>> Hi all,
> > > > >>>>
> > > > >>>> Thanks to Alex for starting this thread - because of this, I’m
> > just
> > > > >>> coming to the realization that OTel Trace and Span IDs are coming
> > built-in
> > > > >>> with Quarkus and my previous work to generate a
> > Request/Correlation ID is
> > > > >>> likely not needed as a result. My original motivation for
> > generation of a
> > > > >>> Request/Correlation ID was to ensure that any client can uniquely
> > identify
> > > > >>> a request made to Polaris, which would be especially useful for
> > debugging
> > > > >>> failing requests or identifying call patterns.
> > > > >>>>
> > > > >>>> As a result, I’m a +1 on Michael’s opinion: we should remove the
> > > > >>> Request/Correlation ID generation and always use the OTel
> > trace/span IDs
> > > > >>> (which come for free with Quarkus) instead for the Correlation ID
> > unless a
> > > > >>> valid header is present, which would take over as the Correlation
> > ID
> > > > >>> instead.
> > > > >>>>
> > > > >>>> —
> > > > >>>>
> > > > >>>> To answer Dmitri’s question re Polaris Events: The intended use
> > case is
> > > > >>> to provide some sort of correlation between events that have
> > occurred as
> > > > >>> part of the same request. For example, if a user makes an
> > CommitTransaction
> > > > >>> request, it would be helpful to see all UpdateTable calls that
> > were made as
> > > > >>> part of that one user request.
> > > > >>>>
> > > > >>>> Best,
> > > > >>>> Adnan Hemani
> > > > >>>>
> > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> > dimas@apache.org>
> > > > >>> wrote:
> > > > >>>>>
> > > > >>>>> Hi Michael,
> > > > >>>>>
> > > > >>>>> Logging x-request-id headers makes sense.
> > > > >>>>>
> > > > >>>>> Just to confirm: if / when we restore that, Polaris will NOT
> > generate
> > > > >>> new
> > > > >>>>> IDs in case the header is not present in the request, correct?
> > > > >>>>>
> > > > >>>>> I believe x-request-id can co-exist with OTel.
> > > > >>>>>
> > > > >>>>> What about adding request IDs to events [1][2]? What's the
> > intended use
> > > > >>>>> case for that? Could you share some context here too?
> > > > >>>>>
> > > > >>>>> Side note: I proposed [2877] flagging event persistence as
> > "beta" in
> > > > >>>>> 1.2.0... This discussion adds another point towards that, I
> > think.
> > > > >>>>>
> > > > >>>>> [1]
> > > > >>>>>
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > > >>>>> [2]
> > > > >>>>>
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > > >>>>>
> > > > >>>>> [2877]
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > > >>>>>
> > > > >>>>> Thanks,
> > > > >>>>> Dmitri.
> > > > >>>>>
> > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > > >>> collado.mike@gmail.com>
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>>> Hey Dmitri
> > > > >>>>>>
> > > > >>>>>> The generating a request id is new code that was added after the
> > > > >>> original
> > > > >>>>>> x-request-id support. You can see the state from ~1 year ago, we
> > > > >>> hard-coded
> > > > >>>>>> request_id as the header we used for the MDC -
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > > >>>>>> . At some point, it was changed to be configurable, then the
> > > > >>>>>> ContextResolverFilter filter was refactored/eliminated and the
> > > > >>>>>> RequestIdFilter took its responsibility, but lost some of its
> > original
> > > > >>>>>> functionality.
> > > > >>>>>>
> > > > >>>>>> My personal opinion is that restoring support for the
> > x-request-id
> > > > >>> header
> > > > >>>>>> is something that we should do, but if the header isn't present,
> > > > >>> falling
> > > > >>>>>> back on simply using OTel trace ids is good enough (better,
> > even) than
> > > > >>>>>> generating another random request id.
> > > > >>>>>>
> > > > >>>>>> Mike
> > > > >>>>>>
> > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > > >>> dimas@apache.org>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Michael,
> > > > >>>>>>>
> > > > >>>>>>> Thanks for the info!
> > > > >>>>>>>
> > > > >>>>>>> Working with Envoy's tracing headers makes sense to me.
> > However, I
> > > > >>>>>> wonder:
> > > > >>>>>>> why would Polaris need to generate a new request ID inside its
> > > > >>> code?..
> > > > >>>>>>> and return it in response headers?
> > > > >>>>>>>
> > > > >>>>>>> How important is it to propagate this ID to Polaris Events?
> > > > >>>>>>>
> > > > >>>>>>> I'm just trying to understand the full context :)
> > > > >>>>>>>
> > > > >>>>>>> Thanks,
> > > > >>>>>>> Dmitri.
> > > > >>>>>>>
> > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > > >>> collado.mike@gmail.com>
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> I think the original intention for this requestId field was to
> > > > >>> support
> > > > >>>>>>>> request propagation from load balancers, like Envoy (
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the default
> > > > >>>>>>>> is Polaris-Request-Id - I think it was originally a custom
> > thing,
> > > > >>> but
> > > > >>>>>>> then
> > > > >>>>>>>> we changed to integrate with existing conventions.
> > Unfortunately,
> > > > >>>>>> looking
> > > > >>>>>>>> through the code, I think that the actual functional plumbing
> > has
> > > > >>> been
> > > > >>>>>>> lost
> > > > >>>>>>>> in the course of multiple refactors around the call context
> > and
> > > > >>>>>>> resolver. I
> > > > >>>>>>>> don't see references to that property or the underlying
> > header.
> > > > >>>>>>>>
> > > > >>>>>>>> Support for the unofficial x-request-id header feels like
> > something
> > > > >>> we
> > > > >>>>>>>> should definitely keep, especially when Polaris is one
> > service in a
> > > > >>>>>> mesh
> > > > >>>>>>> of
> > > > >>>>>>>> services that maybe don't have OTel integration. I'm a fan of
> > the
> > > > >>> OTel
> > > > >>>>>>>> standard, but it's not entirely ubiquitous and there are many
> > > > >>>>>> middleware
> > > > >>>>>>>> layers that know how to forward on the x-request-id header.
> > > > >>>>>>>>
> > > > >>>>>>>> Mike
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> > > > >>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Yes, we should aim for interoperability with the existing
> > de-facto
> > > > >>>>>>>>> standard OTel and make it easy for users to integrate into
> > their
> > > > >>>>>>>>> observability platforms.
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > > >>>>>> dimas@apache.org>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Hi Alex and All,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I certainly support the idea of following OTel standards for
> > > > >>>>>>> achieving
> > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> As to what form the correlation data should take, I believe
> > it is
> > > > >>>>>>>>>> conceptually what the OTel "context" represents. So, I
> > believe it
> > > > >>>>>>> makes
> > > > >>>>>>>>>> sense for Polaris to support standard context propagators
> > at the
> > > > >>>>>> API
> > > > >>>>>>>>> layer.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> If the incoming request has OTel context information, then
> > > > >>>>>> returning
> > > > >>>>>>>> any
> > > > >>>>>>>>>> other "correlation" data in the response is redundant, I
> > think.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> If the incoming request does not have OTel context info,
> > what is
> > > > >>>>>> the
> > > > >>>>>>>>>> purpose of generating a Polaris-specific "correlation ID"?
> > How is
> > > > >>>>>> it
> > > > >>>>>>>>>> envisioned to be used?
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> If the intention is to correlate a Polaris response
> > (operation)
> > > > >>>>>> with
> > > > >>>>>>>>> events
> > > > >>>>>>>>>> that might have resulted from its execution, I believe a
> > more
> > > > >>>>>> robust
> > > > >>>>>>>>>> approach would be to propagate the OTel trace info
> > (starting a new
> > > > >>>>>>>> trace
> > > > >>>>>>>>> if
> > > > >>>>>>>>>> necessary) into event data. Then, Polaris can also return
> > the
> > > > >>> trace
> > > > >>>>>>>>> context
> > > > >>>>>>>>>> in the API response (top span). It's a bit awkward from the
> > OTel
> > > > >>>>>>>>>> perspective, but might be an option for supporting custom
> > > > >>>>>>> correlators.
> > > > >>>>>>>> It
> > > > >>>>>>>>>> could be covered by a feature flag. The header name could be
> > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Custom correlation code will be able to extract the trace
> > ID from
> > > > >>>>>> the
> > > > >>>>>>>>>> response and from events and find related data. Granted, it
> > will
> > > > >>>>>>>> require
> > > > >>>>>>>>> a
> > > > >>>>>>>>>> bit more effort for the custom code to decode trace IDs
> > from the
> > > > >>>>>> OTel
> > > > >>>>>>>>>> context, but the format is standard and not complex. The
> > benefit
> > > > >>>>>> for
> > > > >>>>>>>>>> Polaris, though, is that it can easily integrate with
> > > > >>>>>> OTel-compatible
> > > > >>>>>>>>>> observability platforms regardless of whether any particular
> > > > >>>>>>> deployment
> > > > >>>>>>>>>> uses custom correlators or not.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> WDYT?
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Thanks,
> > > > >>>>>>>>>> Dmitri.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > > >>> adutra@apache.org
> > > > >>>>>>>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> Hi all,
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its
> > purpose is
> > > > >>>>>>> not
> > > > >>>>>>>>>>> entirely clear. It seems to serve as an observability
> > feature to
> > > > >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it to
> > > > >>>>>>>> "correlation
> > > > >>>>>>>>>>> ID" for better alignment with industry standards [1].
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> However, this PR has brought to light overlaps with core
> > > > >>>>>> telemetry
> > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris,
> > each
> > > > >>>>>>>>>>> request already has a trace ID and span ID, making a
> > separate
> > > > >>>>>>>>>>> correlation ID redundant.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris
> > events,
> > > > >>>>>>>>>>> rather than the generated correlation ID, would
> > significantly
> > > > >>>>>>>> simplify
> > > > >>>>>>>>>>> correlation of events with other traces.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> > > > >>>>>>>> correlation
> > > > >>>>>>>>>>> ID for the request, instead of generating a random
> > correlation
> > > > >>>>>> ID.
> > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID
> > header is
> > > > >>>>>>>> present
> > > > >>>>>>>>>>> in the request, use it.
> > > > >>>>>>>>>>> 3. If neither of the above conditions is met, generate a
> > random
> > > > >>>>>>>>>>> correlation ID.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I am somewhat undecided on the best approach when a
> > correlation
> > > > >>>>>> ID
> > > > >>>>>>>>>>> header is present in the request. However, I believe it
> > would be
> > > > >>>>>>> more
> > > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel
> > offers a
> > > > >>>>>> more
> > > > >>>>>>>>>>> robust solution for client-to-server trace propagation,
> > e.g. W3C
> > > > >>>>>>>> Trace
> > > > >>>>>>>>>>> Context propagation [2].
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Please share your thoughts!
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>> Alex
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> [1]:
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > > >>>>>>>>>>> [2]:
> > > > >>>
> > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > >
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi
Yeah it looks good to me and I agree we have a consensus.
Thanks !
Regards
JB
Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <ad...@apache.org> a écrit :
> Hi all,
>
> Thanks for the good discussion so far. I think we have a broad consensus.
>
> Unless anybody disagrees I am going to move forward with the the
> following revised plan, incorporating the latest feedback:
>
> 1. Restore the original functionality for request IDs, change the
> default header name back to x-request-id
>
> 2. Remove RequestIdGenerator and related functionality.
>
> 3. Update PolarisEvent, expose request ID if available, expose OTel
> context if available.
>
> 4. Update events table SQL schema: insert request ID if available,
> insert OTel context if available.
>
> Thanks,
> Alex
>
> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >
> > Hi
> >
> > I think it's better to use w3c_trace_context (better than two fields).
> >
> > Regarding x-request-id, I think it's for a different purpose,
> > specifically for persistence. Personally, I think we can achieve the
> > same with span and events.
> > If we want to "distinguish" the two layers, it makes sense to have
> > x-request-id (not sure it's super helpful). Else, I think we can
> > achieve the same with span/event.
> >
> > Just my $0.01
> >
> > Regards
> > JB
> >
> > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> > <ad...@snowflake.com.invalid> wrote:
> > >
> > > Hi all,
> > >
> > > +1 to all of Alex’s AIs with Dmitri’s suggested changes as well. Great
> find, Dmitri!
> > >
> > > I’m still debating with myself on whether we need to store the
> `x-request-id` field as part of the Events persistence. Can we think of a
> good use case where this is more helpful to the user than the OTel
> Trace/Span IDs? I am making the assumption here that those are still being
> returned back to the client.
> > >
> > > Best,
> > > Adnan Hemani
> > >
> > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org>
> wrote:
> > > >
> > > > Hi Dmitri,
> > > >
> > > > Yes, I think your suggestion to use just one field w3c_trace_context
> > > > makes more sense than two fields (span_id and trace_id).
> > > >
> > > > With that, I also think that we are slowly drifting into
> > > > implementation considerations; let's get consensus on the general
> > > > design first, and we can certainly fine-tune the actual Java methods
> > > > and SQL table columns in the future PR. WDYT?
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <
> dimas@apache.org> wrote:
> > > >>
> > > >> Hi All,
> > > >>
> > > >> Many thanks for the background info, Adnan!
> > > >>
> > > >> +1 to action items proposed by Alex!
> > > >>
> > > >> Re: (3) Can we abstract request/trace info into a separate object
> without
> > > >> exposing those accessors on the Event class directly? OTel defines
> > > >> trace/span concepts, but in request ID is a bit foreign to OTel.
> Having
> > > >> tracing / request ID isolated in java could help with maintaining
> it and
> > > >> potentially supporting other (custom) tracing methods.
> > > >>
> > > >> Re: (4) I'd like to propose storing OTel correlation data in the
> form of a
> > > >> standard context propagation string, e.g. W3C trace-context [1]
> (same value
> > > >> as its HTTP header), so the column could be called
> w3c_trace_context or
> > > >> simply trace_context.
> > > >>
> > > >> Open question: do we need to write a separate, individual trace ID
> field in
> > > >> SQL? I suppose it is not very useful since correlating it to other
> trace
> > > >> data already requires understanding OTel context propagation and a
> query
> > > >> against trace_context can still be made using string-matching
> clauses. We
> > > >> could probably (additionally) store it in the request_id column if
> the
> > > >> Polaris-specific request ID header is not set.
> > > >>
> > > >> As for span ID, I do not really see a use case for persisting it
> > > >> individually. It is very specific to OTel trace data construction.
> > > >>
> > > >> Actually, using the W3C trace context [1] encoding probably makes
> sense in
> > > >> the java event representation too. Interested callers can easily
> decode
> > > >> this information since the format is well-defined. As a side
> benefit, this
> > > >> opens opportunities for downstream event consumers to connect
> (propagate
> > > >> context) to traces that produced events based on the event data
> itself,
> > > >> without relying on the intermediate frameworks. This may be
> desirable since
> > > >> the current Polaris event persistence impl. writes events in
> batches, so
> > > >> the association to individual requests that produced those events
> is lost.
> > > >> Whether to perform this kind of context propagation will be at the
> > > >> discretion of the event consumer, of course (outside of Polaris
> code).
> > > >>
> > > >> [1]
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > > >>
> > > >> WDYT?
> > > >>
> > > >> Thanks,
> > > >> Dmitri.
> > > >>
> > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> > > >>
> > > >>> Hi all,
> > > >>>
> > > >>> Thank you for chiming in; the context around request IDs is now
> clear.
> > > >>>
> > > >>> Trying to summarize this thread into actionable items, here's what
> I
> > > >>> propose:
> > > >>>
> > > >>> 1. Restore the original functionality for request IDs.
> > > >>> - Change the default header name back to x-request-id (despite
> the
> > > >>> x- prefix being deprecated), but keep it configurable as today.
> > > >>> 2. Remove RequestIdGenerator and related functionality.
> > > >>> - Do not generate a request ID if the REST client doesn't
> provide one.
> > > >>> 3. Update PolarisEvent:
> > > >>> - Expose new requestId(), traceId(), spanId() methods, all
> nullable.
> > > >>> - This would align with the emerging consensus around including
> > > >>> contextual information in PolarisEvent [1].
> > > >>> 4. Update events table SQL schema:
> > > >>> - Insert the client-provided request ID into the request_id
> > > >>> column; otherwise, insert null.
> > > >>> - Add two new nullable columns, trace_id and span_id, and
> populate
> > > >>> them if OTel is enabled.
> > > >>>
> > > >>> From our discussions, I think it's important to not conflate OTel
> > > >>> tracing fields with Envoy tracing fields, which is why I suggest we
> > > >>> use separate fields / columns for them.
> > > >>>
> > > >>> Would the above plan work for everyone?
> > > >>>
> > > >>> Thanks,
> > > >>> Alex
> > > >>>
> > > >>> [1]:
> https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > > >>>
> > > >>>
> > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > > >>> <ad...@snowflake.com.invalid> wrote:
> > > >>>>
> > > >>>> Hi all,
> > > >>>>
> > > >>>> Thanks to Alex for starting this thread - because of this, I’m
> just
> > > >>> coming to the realization that OTel Trace and Span IDs are coming
> built-in
> > > >>> with Quarkus and my previous work to generate a
> Request/Correlation ID is
> > > >>> likely not needed as a result. My original motivation for
> generation of a
> > > >>> Request/Correlation ID was to ensure that any client can uniquely
> identify
> > > >>> a request made to Polaris, which would be especially useful for
> debugging
> > > >>> failing requests or identifying call patterns.
> > > >>>>
> > > >>>> As a result, I’m a +1 on Michael’s opinion: we should remove the
> > > >>> Request/Correlation ID generation and always use the OTel
> trace/span IDs
> > > >>> (which come for free with Quarkus) instead for the Correlation ID
> unless a
> > > >>> valid header is present, which would take over as the Correlation
> ID
> > > >>> instead.
> > > >>>>
> > > >>>> —
> > > >>>>
> > > >>>> To answer Dmitri’s question re Polaris Events: The intended use
> case is
> > > >>> to provide some sort of correlation between events that have
> occurred as
> > > >>> part of the same request. For example, if a user makes an
> CommitTransaction
> > > >>> request, it would be helpful to see all UpdateTable calls that
> were made as
> > > >>> part of that one user request.
> > > >>>>
> > > >>>> Best,
> > > >>>> Adnan Hemani
> > > >>>>
> > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <
> dimas@apache.org>
> > > >>> wrote:
> > > >>>>>
> > > >>>>> Hi Michael,
> > > >>>>>
> > > >>>>> Logging x-request-id headers makes sense.
> > > >>>>>
> > > >>>>> Just to confirm: if / when we restore that, Polaris will NOT
> generate
> > > >>> new
> > > >>>>> IDs in case the header is not present in the request, correct?
> > > >>>>>
> > > >>>>> I believe x-request-id can co-exist with OTel.
> > > >>>>>
> > > >>>>> What about adding request IDs to events [1][2]? What's the
> intended use
> > > >>>>> case for that? Could you share some context here too?
> > > >>>>>
> > > >>>>> Side note: I proposed [2877] flagging event persistence as
> "beta" in
> > > >>>>> 1.2.0... This discussion adds another point towards that, I
> think.
> > > >>>>>
> > > >>>>> [1]
> > > >>>>>
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > > >>>>> [2]
> > > >>>>>
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > > >>>>>
> > > >>>>> [2877]
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > > >>>>>
> > > >>>>> Thanks,
> > > >>>>> Dmitri.
> > > >>>>>
> > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > > >>> collado.mike@gmail.com>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> Hey Dmitri
> > > >>>>>>
> > > >>>>>> The generating a request id is new code that was added after the
> > > >>> original
> > > >>>>>> x-request-id support. You can see the state from ~1 year ago, we
> > > >>> hard-coded
> > > >>>>>> request_id as the header we used for the MDC -
> > > >>>>>>
> > > >>>>>>
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > > >>>>>> . At some point, it was changed to be configurable, then the
> > > >>>>>> ContextResolverFilter filter was refactored/eliminated and the
> > > >>>>>> RequestIdFilter took its responsibility, but lost some of its
> original
> > > >>>>>> functionality.
> > > >>>>>>
> > > >>>>>> My personal opinion is that restoring support for the
> x-request-id
> > > >>> header
> > > >>>>>> is something that we should do, but if the header isn't present,
> > > >>> falling
> > > >>>>>> back on simply using OTel trace ids is good enough (better,
> even) than
> > > >>>>>> generating another random request id.
> > > >>>>>>
> > > >>>>>> Mike
> > > >>>>>>
> > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > > >>> dimas@apache.org>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Michael,
> > > >>>>>>>
> > > >>>>>>> Thanks for the info!
> > > >>>>>>>
> > > >>>>>>> Working with Envoy's tracing headers makes sense to me.
> However, I
> > > >>>>>> wonder:
> > > >>>>>>> why would Polaris need to generate a new request ID inside its
> > > >>> code?..
> > > >>>>>>> and return it in response headers?
> > > >>>>>>>
> > > >>>>>>> How important is it to propagate this ID to Polaris Events?
> > > >>>>>>>
> > > >>>>>>> I'm just trying to understand the full context :)
> > > >>>>>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Dmitri.
> > > >>>>>>>
> > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > > >>> collado.mike@gmail.com>
> > > >>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> I think the original intention for this requestId field was to
> > > >>> support
> > > >>>>>>>> request propagation from load balancers, like Envoy (
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the default
> > > >>>>>>>> is Polaris-Request-Id - I think it was originally a custom
> thing,
> > > >>> but
> > > >>>>>>> then
> > > >>>>>>>> we changed to integrate with existing conventions.
> Unfortunately,
> > > >>>>>> looking
> > > >>>>>>>> through the code, I think that the actual functional plumbing
> has
> > > >>> been
> > > >>>>>>> lost
> > > >>>>>>>> in the course of multiple refactors around the call context
> and
> > > >>>>>>> resolver. I
> > > >>>>>>>> don't see references to that property or the underlying
> header.
> > > >>>>>>>>
> > > >>>>>>>> Support for the unofficial x-request-id header feels like
> something
> > > >>> we
> > > >>>>>>>> should definitely keep, especially when Polaris is one
> service in a
> > > >>>>>> mesh
> > > >>>>>>> of
> > > >>>>>>>> services that maybe don't have OTel integration. I'm a fan of
> the
> > > >>> OTel
> > > >>>>>>>> standard, but it's not entirely ubiquitous and there are many
> > > >>>>>> middleware
> > > >>>>>>>> layers that know how to forward on the x-request-id header.
> > > >>>>>>>>
> > > >>>>>>>> Mike
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> > > >>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Yes, we should aim for interoperability with the existing
> de-facto
> > > >>>>>>>>> standard OTel and make it easy for users to integrate into
> their
> > > >>>>>>>>> observability platforms.
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > >>>>>> dimas@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hi Alex and All,
> > > >>>>>>>>>>
> > > >>>>>>>>>> I certainly support the idea of following OTel standards for
> > > >>>>>>> achieving
> > > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > > >>>>>>>>>>
> > > >>>>>>>>>> As to what form the correlation data should take, I believe
> it is
> > > >>>>>>>>>> conceptually what the OTel "context" represents. So, I
> believe it
> > > >>>>>>> makes
> > > >>>>>>>>>> sense for Polaris to support standard context propagators
> at the
> > > >>>>>> API
> > > >>>>>>>>> layer.
> > > >>>>>>>>>>
> > > >>>>>>>>>> If the incoming request has OTel context information, then
> > > >>>>>> returning
> > > >>>>>>>> any
> > > >>>>>>>>>> other "correlation" data in the response is redundant, I
> think.
> > > >>>>>>>>>>
> > > >>>>>>>>>> If the incoming request does not have OTel context info,
> what is
> > > >>>>>> the
> > > >>>>>>>>>> purpose of generating a Polaris-specific "correlation ID"?
> How is
> > > >>>>>> it
> > > >>>>>>>>>> envisioned to be used?
> > > >>>>>>>>>>
> > > >>>>>>>>>> If the intention is to correlate a Polaris response
> (operation)
> > > >>>>>> with
> > > >>>>>>>>> events
> > > >>>>>>>>>> that might have resulted from its execution, I believe a
> more
> > > >>>>>> robust
> > > >>>>>>>>>> approach would be to propagate the OTel trace info
> (starting a new
> > > >>>>>>>> trace
> > > >>>>>>>>> if
> > > >>>>>>>>>> necessary) into event data. Then, Polaris can also return
> the
> > > >>> trace
> > > >>>>>>>>> context
> > > >>>>>>>>>> in the API response (top span). It's a bit awkward from the
> OTel
> > > >>>>>>>>>> perspective, but might be an option for supporting custom
> > > >>>>>>> correlators.
> > > >>>>>>>> It
> > > >>>>>>>>>> could be covered by a feature flag. The header name could be
> > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Custom correlation code will be able to extract the trace
> ID from
> > > >>>>>> the
> > > >>>>>>>>>> response and from events and find related data. Granted, it
> will
> > > >>>>>>>> require
> > > >>>>>>>>> a
> > > >>>>>>>>>> bit more effort for the custom code to decode trace IDs
> from the
> > > >>>>>> OTel
> > > >>>>>>>>>> context, but the format is standard and not complex. The
> benefit
> > > >>>>>> for
> > > >>>>>>>>>> Polaris, though, is that it can easily integrate with
> > > >>>>>> OTel-compatible
> > > >>>>>>>>>> observability platforms regardless of whether any particular
> > > >>>>>>> deployment
> > > >>>>>>>>>> uses custom correlators or not.
> > > >>>>>>>>>>
> > > >>>>>>>>>> WDYT?
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thanks,
> > > >>>>>>>>>> Dmitri.
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > > >>> adutra@apache.org
> > > >>>>>>>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Hi all,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its
> purpose is
> > > >>>>>>> not
> > > >>>>>>>>>>> entirely clear. It seems to serve as an observability
> feature to
> > > >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it to
> > > >>>>>>>> "correlation
> > > >>>>>>>>>>> ID" for better alignment with industry standards [1].
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> However, this PR has brought to light overlaps with core
> > > >>>>>> telemetry
> > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris,
> each
> > > >>>>>>>>>>> request already has a trace ID and span ID, making a
> separate
> > > >>>>>>>>>>> correlation ID redundant.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris
> events,
> > > >>>>>>>>>>> rather than the generated correlation ID, would
> significantly
> > > >>>>>>>> simplify
> > > >>>>>>>>>>> correlation of events with other traces.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Therefore, I propose the following changes:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> > > >>>>>>>> correlation
> > > >>>>>>>>>>> ID for the request, instead of generating a random
> correlation
> > > >>>>>> ID.
> > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID
> header is
> > > >>>>>>>> present
> > > >>>>>>>>>>> in the request, use it.
> > > >>>>>>>>>>> 3. If neither of the above conditions is met, generate a
> random
> > > >>>>>>>>>>> correlation ID.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I am somewhat undecided on the best approach when a
> correlation
> > > >>>>>> ID
> > > >>>>>>>>>>> header is present in the request. However, I believe it
> would be
> > > >>>>>>> more
> > > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel
> offers a
> > > >>>>>> more
> > > >>>>>>>>>>> robust solution for client-to-server trace propagation,
> e.g. W3C
> > > >>>>>>>> Trace
> > > >>>>>>>>>>> Context propagation [2].
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Please share your thoughts!
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks,
> > > >>>>>>>>>>> Alex
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> [1]:
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > > >>>>>>>>>>> [2]:
> > > >>>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,
Thanks for the good discussion so far. I think we have a broad consensus.
Unless anybody disagrees I am going to move forward with the the
following revised plan, incorporating the latest feedback:
1. Restore the original functionality for request IDs, change the
default header name back to x-request-id
2. Remove RequestIdGenerator and related functionality.
3. Update PolarisEvent, expose request ID if available, expose OTel
context if available.
4. Update events table SQL schema: insert request ID if available,
insert OTel context if available.
Thanks,
Alex
On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi
>
> I think it's better to use w3c_trace_context (better than two fields).
>
> Regarding x-request-id, I think it's for a different purpose,
> specifically for persistence. Personally, I think we can achieve the
> same with span and events.
> If we want to "distinguish" the two layers, it makes sense to have
> x-request-id (not sure it's super helpful). Else, I think we can
> achieve the same with span/event.
>
> Just my $0.01
>
> Regards
> JB
>
> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
> <ad...@snowflake.com.invalid> wrote:
> >
> > Hi all,
> >
> > +1 to all of Alex’s AIs with Dmitri’s suggested changes as well. Great find, Dmitri!
> >
> > I’m still debating with myself on whether we need to store the `x-request-id` field as part of the Events persistence. Can we think of a good use case where this is more helpful to the user than the OTel Trace/Span IDs? I am making the assumption here that those are still being returned back to the client.
> >
> > Best,
> > Adnan Hemani
> >
> > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org> wrote:
> > >
> > > Hi Dmitri,
> > >
> > > Yes, I think your suggestion to use just one field w3c_trace_context
> > > makes more sense than two fields (span_id and trace_id).
> > >
> > > With that, I also think that we are slowly drifting into
> > > implementation considerations; let's get consensus on the general
> > > design first, and we can certainly fine-tune the actual Java methods
> > > and SQL table columns in the future PR. WDYT?
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
> > >>
> > >> Hi All,
> > >>
> > >> Many thanks for the background info, Adnan!
> > >>
> > >> +1 to action items proposed by Alex!
> > >>
> > >> Re: (3) Can we abstract request/trace info into a separate object without
> > >> exposing those accessors on the Event class directly? OTel defines
> > >> trace/span concepts, but in request ID is a bit foreign to OTel. Having
> > >> tracing / request ID isolated in java could help with maintaining it and
> > >> potentially supporting other (custom) tracing methods.
> > >>
> > >> Re: (4) I'd like to propose storing OTel correlation data in the form of a
> > >> standard context propagation string, e.g. W3C trace-context [1] (same value
> > >> as its HTTP header), so the column could be called w3c_trace_context or
> > >> simply trace_context.
> > >>
> > >> Open question: do we need to write a separate, individual trace ID field in
> > >> SQL? I suppose it is not very useful since correlating it to other trace
> > >> data already requires understanding OTel context propagation and a query
> > >> against trace_context can still be made using string-matching clauses. We
> > >> could probably (additionally) store it in the request_id column if the
> > >> Polaris-specific request ID header is not set.
> > >>
> > >> As for span ID, I do not really see a use case for persisting it
> > >> individually. It is very specific to OTel trace data construction.
> > >>
> > >> Actually, using the W3C trace context [1] encoding probably makes sense in
> > >> the java event representation too. Interested callers can easily decode
> > >> this information since the format is well-defined. As a side benefit, this
> > >> opens opportunities for downstream event consumers to connect (propagate
> > >> context) to traces that produced events based on the event data itself,
> > >> without relying on the intermediate frameworks. This may be desirable since
> > >> the current Polaris event persistence impl. writes events in batches, so
> > >> the association to individual requests that produced those events is lost.
> > >> Whether to perform this kind of context propagation will be at the
> > >> discretion of the event consumer, of course (outside of Polaris code).
> > >>
> > >> [1] https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> > >>
> > >> WDYT?
> > >>
> > >> Thanks,
> > >> Dmitri.
> > >>
> > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org> wrote:
> > >>
> > >>> Hi all,
> > >>>
> > >>> Thank you for chiming in; the context around request IDs is now clear.
> > >>>
> > >>> Trying to summarize this thread into actionable items, here's what I
> > >>> propose:
> > >>>
> > >>> 1. Restore the original functionality for request IDs.
> > >>> - Change the default header name back to x-request-id (despite the
> > >>> x- prefix being deprecated), but keep it configurable as today.
> > >>> 2. Remove RequestIdGenerator and related functionality.
> > >>> - Do not generate a request ID if the REST client doesn't provide one.
> > >>> 3. Update PolarisEvent:
> > >>> - Expose new requestId(), traceId(), spanId() methods, all nullable.
> > >>> - This would align with the emerging consensus around including
> > >>> contextual information in PolarisEvent [1].
> > >>> 4. Update events table SQL schema:
> > >>> - Insert the client-provided request ID into the request_id
> > >>> column; otherwise, insert null.
> > >>> - Add two new nullable columns, trace_id and span_id, and populate
> > >>> them if OTel is enabled.
> > >>>
> > >>> From our discussions, I think it's important to not conflate OTel
> > >>> tracing fields with Envoy tracing fields, which is why I suggest we
> > >>> use separate fields / columns for them.
> > >>>
> > >>> Would the above plan work for everyone?
> > >>>
> > >>> Thanks,
> > >>> Alex
> > >>>
> > >>> [1]: https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> > >>>
> > >>>
> > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > >>> <ad...@snowflake.com.invalid> wrote:
> > >>>>
> > >>>> Hi all,
> > >>>>
> > >>>> Thanks to Alex for starting this thread - because of this, I’m just
> > >>> coming to the realization that OTel Trace and Span IDs are coming built-in
> > >>> with Quarkus and my previous work to generate a Request/Correlation ID is
> > >>> likely not needed as a result. My original motivation for generation of a
> > >>> Request/Correlation ID was to ensure that any client can uniquely identify
> > >>> a request made to Polaris, which would be especially useful for debugging
> > >>> failing requests or identifying call patterns.
> > >>>>
> > >>>> As a result, I’m a +1 on Michael’s opinion: we should remove the
> > >>> Request/Correlation ID generation and always use the OTel trace/span IDs
> > >>> (which come for free with Quarkus) instead for the Correlation ID unless a
> > >>> valid header is present, which would take over as the Correlation ID
> > >>> instead.
> > >>>>
> > >>>> —
> > >>>>
> > >>>> To answer Dmitri’s question re Polaris Events: The intended use case is
> > >>> to provide some sort of correlation between events that have occurred as
> > >>> part of the same request. For example, if a user makes an CommitTransaction
> > >>> request, it would be helpful to see all UpdateTable calls that were made as
> > >>> part of that one user request.
> > >>>>
> > >>>> Best,
> > >>>> Adnan Hemani
> > >>>>
> > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org>
> > >>> wrote:
> > >>>>>
> > >>>>> Hi Michael,
> > >>>>>
> > >>>>> Logging x-request-id headers makes sense.
> > >>>>>
> > >>>>> Just to confirm: if / when we restore that, Polaris will NOT generate
> > >>> new
> > >>>>> IDs in case the header is not present in the request, correct?
> > >>>>>
> > >>>>> I believe x-request-id can co-exist with OTel.
> > >>>>>
> > >>>>> What about adding request IDs to events [1][2]? What's the intended use
> > >>>>> case for that? Could you share some context here too?
> > >>>>>
> > >>>>> Side note: I proposed [2877] flagging event persistence as "beta" in
> > >>>>> 1.2.0... This discussion adds another point towards that, I think.
> > >>>>>
> > >>>>> [1]
> > >>>>>
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> > >>>>> [2]
> > >>>>>
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> > >>>>>
> > >>>>> [2877]
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Dmitri.
> > >>>>>
> > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > >>> collado.mike@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hey Dmitri
> > >>>>>>
> > >>>>>> The generating a request id is new code that was added after the
> > >>> original
> > >>>>>> x-request-id support. You can see the state from ~1 year ago, we
> > >>> hard-coded
> > >>>>>> request_id as the header we used for the MDC -
> > >>>>>>
> > >>>>>>
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> > >>>>>> . At some point, it was changed to be configurable, then the
> > >>>>>> ContextResolverFilter filter was refactored/eliminated and the
> > >>>>>> RequestIdFilter took its responsibility, but lost some of its original
> > >>>>>> functionality.
> > >>>>>>
> > >>>>>> My personal opinion is that restoring support for the x-request-id
> > >>> header
> > >>>>>> is something that we should do, but if the header isn't present,
> > >>> falling
> > >>>>>> back on simply using OTel trace ids is good enough (better, even) than
> > >>>>>> generating another random request id.
> > >>>>>>
> > >>>>>> Mike
> > >>>>>>
> > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > >>> dimas@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Michael,
> > >>>>>>>
> > >>>>>>> Thanks for the info!
> > >>>>>>>
> > >>>>>>> Working with Envoy's tracing headers makes sense to me. However, I
> > >>>>>> wonder:
> > >>>>>>> why would Polaris need to generate a new request ID inside its
> > >>> code?..
> > >>>>>>> and return it in response headers?
> > >>>>>>>
> > >>>>>>> How important is it to propagate this ID to Polaris Events?
> > >>>>>>>
> > >>>>>>> I'm just trying to understand the full context :)
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Dmitri.
> > >>>>>>>
> > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > >>> collado.mike@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> I think the original intention for this requestId field was to
> > >>> support
> > >>>>>>>> request propagation from load balancers, like Envoy (
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> > >>>>>>>> ), which is distinct from OTEL. Don't ask me why the default
> > >>>>>>>> is Polaris-Request-Id - I think it was originally a custom thing,
> > >>> but
> > >>>>>>> then
> > >>>>>>>> we changed to integrate with existing conventions. Unfortunately,
> > >>>>>> looking
> > >>>>>>>> through the code, I think that the actual functional plumbing has
> > >>> been
> > >>>>>>> lost
> > >>>>>>>> in the course of multiple refactors around the call context and
> > >>>>>>> resolver. I
> > >>>>>>>> don't see references to that property or the underlying header.
> > >>>>>>>>
> > >>>>>>>> Support for the unofficial x-request-id header feels like something
> > >>> we
> > >>>>>>>> should definitely keep, especially when Polaris is one service in a
> > >>>>>> mesh
> > >>>>>>> of
> > >>>>>>>> services that maybe don't have OTel integration. I'm a fan of the
> > >>> OTel
> > >>>>>>>> standard, but it's not entirely ubiquitous and there are many
> > >>>>>> middleware
> > >>>>>>>> layers that know how to forward on the x-request-id header.
> > >>>>>>>>
> > >>>>>>>> Mike
> > >>>>>>>>
> > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> > >>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Yes, we should aim for interoperability with the existing de-facto
> > >>>>>>>>> standard OTel and make it easy for users to integrate into their
> > >>>>>>>>> observability platforms.
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > >>>>>> dimas@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Alex and All,
> > >>>>>>>>>>
> > >>>>>>>>>> I certainly support the idea of following OTel standards for
> > >>>>>>> achieving
> > >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> > >>>>>>>>>>
> > >>>>>>>>>> As to what form the correlation data should take, I believe it is
> > >>>>>>>>>> conceptually what the OTel "context" represents. So, I believe it
> > >>>>>>> makes
> > >>>>>>>>>> sense for Polaris to support standard context propagators at the
> > >>>>>> API
> > >>>>>>>>> layer.
> > >>>>>>>>>>
> > >>>>>>>>>> If the incoming request has OTel context information, then
> > >>>>>> returning
> > >>>>>>>> any
> > >>>>>>>>>> other "correlation" data in the response is redundant, I think.
> > >>>>>>>>>>
> > >>>>>>>>>> If the incoming request does not have OTel context info, what is
> > >>>>>> the
> > >>>>>>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
> > >>>>>> it
> > >>>>>>>>>> envisioned to be used?
> > >>>>>>>>>>
> > >>>>>>>>>> If the intention is to correlate a Polaris response (operation)
> > >>>>>> with
> > >>>>>>>>> events
> > >>>>>>>>>> that might have resulted from its execution, I believe a more
> > >>>>>> robust
> > >>>>>>>>>> approach would be to propagate the OTel trace info (starting a new
> > >>>>>>>> trace
> > >>>>>>>>> if
> > >>>>>>>>>> necessary) into event data. Then, Polaris can also return the
> > >>> trace
> > >>>>>>>>> context
> > >>>>>>>>>> in the API response (top span). It's a bit awkward from the OTel
> > >>>>>>>>>> perspective, but might be an option for supporting custom
> > >>>>>>> correlators.
> > >>>>>>>> It
> > >>>>>>>>>> could be covered by a feature flag. The header name could be
> > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> > >>>>>>>>>>
> > >>>>>>>>>> Custom correlation code will be able to extract the trace ID from
> > >>>>>> the
> > >>>>>>>>>> response and from events and find related data. Granted, it will
> > >>>>>>>> require
> > >>>>>>>>> a
> > >>>>>>>>>> bit more effort for the custom code to decode trace IDs from the
> > >>>>>> OTel
> > >>>>>>>>>> context, but the format is standard and not complex. The benefit
> > >>>>>> for
> > >>>>>>>>>> Polaris, though, is that it can easily integrate with
> > >>>>>> OTel-compatible
> > >>>>>>>>>> observability platforms regardless of whether any particular
> > >>>>>>> deployment
> > >>>>>>>>>> uses custom correlators or not.
> > >>>>>>>>>>
> > >>>>>>>>>> WDYT?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Dmitri.
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > >>> adutra@apache.org
> > >>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi all,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
> > >>>>>>> not
> > >>>>>>>>>>> entirely clear. It seems to serve as an observability feature to
> > >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it to
> > >>>>>>>> "correlation
> > >>>>>>>>>>> ID" for better alignment with industry standards [1].
> > >>>>>>>>>>>
> > >>>>>>>>>>> However, this PR has brought to light overlaps with core
> > >>>>>> telemetry
> > >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > >>>>>>>>>>> request already has a trace ID and span ID, making a separate
> > >>>>>>>>>>> correlation ID redundant.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
> > >>>>>>>>>>> rather than the generated correlation ID, would significantly
> > >>>>>>>> simplify
> > >>>>>>>>>>> correlation of events with other traces.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Therefore, I propose the following changes:
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> > >>>>>>>> correlation
> > >>>>>>>>>>> ID for the request, instead of generating a random correlation
> > >>>>>> ID.
> > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
> > >>>>>>>> present
> > >>>>>>>>>>> in the request, use it.
> > >>>>>>>>>>> 3. If neither of the above conditions is met, generate a random
> > >>>>>>>>>>> correlation ID.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I am somewhat undecided on the best approach when a correlation
> > >>>>>> ID
> > >>>>>>>>>>> header is present in the request. However, I believe it would be
> > >>>>>>> more
> > >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
> > >>>>>> more
> > >>>>>>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
> > >>>>>>>> Trace
> > >>>>>>>>>>> Context propagation [2].
> > >>>>>>>>>>>
> > >>>>>>>>>>> Please share your thoughts!
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Alex
> > >>>>>>>>>>>
> > >>>>>>>>>>> [1]:
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> > >>>>>>>>>>> [2]:
> > >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi
I think it's better to use w3c_trace_context (better than two fields).
Regarding x-request-id, I think it's for a different purpose,
specifically for persistence. Personally, I think we can achieve the
same with span and events.
If we want to "distinguish" the two layers, it makes sense to have
x-request-id (not sure it's super helpful). Else, I think we can
achieve the same with span/event.
Just my $0.01
Regards
JB
On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani
<ad...@snowflake.com.invalid> wrote:
>
> Hi all,
>
> +1 to all of Alex’s AIs with Dmitri’s suggested changes as well. Great find, Dmitri!
>
> I’m still debating with myself on whether we need to store the `x-request-id` field as part of the Events persistence. Can we think of a good use case where this is more helpful to the user than the OTel Trace/Span IDs? I am making the assumption here that those are still being returned back to the client.
>
> Best,
> Adnan Hemani
>
> > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org> wrote:
> >
> > Hi Dmitri,
> >
> > Yes, I think your suggestion to use just one field w3c_trace_context
> > makes more sense than two fields (span_id and trace_id).
> >
> > With that, I also think that we are slowly drifting into
> > implementation considerations; let's get consensus on the general
> > design first, and we can certainly fine-tune the actual Java methods
> > and SQL table columns in the future PR. WDYT?
> >
> > Thanks,
> > Alex
> >
> > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
> >>
> >> Hi All,
> >>
> >> Many thanks for the background info, Adnan!
> >>
> >> +1 to action items proposed by Alex!
> >>
> >> Re: (3) Can we abstract request/trace info into a separate object without
> >> exposing those accessors on the Event class directly? OTel defines
> >> trace/span concepts, but in request ID is a bit foreign to OTel. Having
> >> tracing / request ID isolated in java could help with maintaining it and
> >> potentially supporting other (custom) tracing methods.
> >>
> >> Re: (4) I'd like to propose storing OTel correlation data in the form of a
> >> standard context propagation string, e.g. W3C trace-context [1] (same value
> >> as its HTTP header), so the column could be called w3c_trace_context or
> >> simply trace_context.
> >>
> >> Open question: do we need to write a separate, individual trace ID field in
> >> SQL? I suppose it is not very useful since correlating it to other trace
> >> data already requires understanding OTel context propagation and a query
> >> against trace_context can still be made using string-matching clauses. We
> >> could probably (additionally) store it in the request_id column if the
> >> Polaris-specific request ID header is not set.
> >>
> >> As for span ID, I do not really see a use case for persisting it
> >> individually. It is very specific to OTel trace data construction.
> >>
> >> Actually, using the W3C trace context [1] encoding probably makes sense in
> >> the java event representation too. Interested callers can easily decode
> >> this information since the format is well-defined. As a side benefit, this
> >> opens opportunities for downstream event consumers to connect (propagate
> >> context) to traces that produced events based on the event data itself,
> >> without relying on the intermediate frameworks. This may be desirable since
> >> the current Polaris event persistence impl. writes events in batches, so
> >> the association to individual requests that produced those events is lost.
> >> Whether to perform this kind of context propagation will be at the
> >> discretion of the event consumer, of course (outside of Polaris code).
> >>
> >> [1] https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
> >>
> >> WDYT?
> >>
> >> Thanks,
> >> Dmitri.
> >>
> >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org> wrote:
> >>
> >>> Hi all,
> >>>
> >>> Thank you for chiming in; the context around request IDs is now clear.
> >>>
> >>> Trying to summarize this thread into actionable items, here's what I
> >>> propose:
> >>>
> >>> 1. Restore the original functionality for request IDs.
> >>> - Change the default header name back to x-request-id (despite the
> >>> x- prefix being deprecated), but keep it configurable as today.
> >>> 2. Remove RequestIdGenerator and related functionality.
> >>> - Do not generate a request ID if the REST client doesn't provide one.
> >>> 3. Update PolarisEvent:
> >>> - Expose new requestId(), traceId(), spanId() methods, all nullable.
> >>> - This would align with the emerging consensus around including
> >>> contextual information in PolarisEvent [1].
> >>> 4. Update events table SQL schema:
> >>> - Insert the client-provided request ID into the request_id
> >>> column; otherwise, insert null.
> >>> - Add two new nullable columns, trace_id and span_id, and populate
> >>> them if OTel is enabled.
> >>>
> >>> From our discussions, I think it's important to not conflate OTel
> >>> tracing fields with Envoy tracing fields, which is why I suggest we
> >>> use separate fields / columns for them.
> >>>
> >>> Would the above plan work for everyone?
> >>>
> >>> Thanks,
> >>> Alex
> >>>
> >>> [1]: https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
> >>>
> >>>
> >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> >>> <ad...@snowflake.com.invalid> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> Thanks to Alex for starting this thread - because of this, I’m just
> >>> coming to the realization that OTel Trace and Span IDs are coming built-in
> >>> with Quarkus and my previous work to generate a Request/Correlation ID is
> >>> likely not needed as a result. My original motivation for generation of a
> >>> Request/Correlation ID was to ensure that any client can uniquely identify
> >>> a request made to Polaris, which would be especially useful for debugging
> >>> failing requests or identifying call patterns.
> >>>>
> >>>> As a result, I’m a +1 on Michael’s opinion: we should remove the
> >>> Request/Correlation ID generation and always use the OTel trace/span IDs
> >>> (which come for free with Quarkus) instead for the Correlation ID unless a
> >>> valid header is present, which would take over as the Correlation ID
> >>> instead.
> >>>>
> >>>> —
> >>>>
> >>>> To answer Dmitri’s question re Polaris Events: The intended use case is
> >>> to provide some sort of correlation between events that have occurred as
> >>> part of the same request. For example, if a user makes an CommitTransaction
> >>> request, it would be helpful to see all UpdateTable calls that were made as
> >>> part of that one user request.
> >>>>
> >>>> Best,
> >>>> Adnan Hemani
> >>>>
> >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org>
> >>> wrote:
> >>>>>
> >>>>> Hi Michael,
> >>>>>
> >>>>> Logging x-request-id headers makes sense.
> >>>>>
> >>>>> Just to confirm: if / when we restore that, Polaris will NOT generate
> >>> new
> >>>>> IDs in case the header is not present in the request, correct?
> >>>>>
> >>>>> I believe x-request-id can co-exist with OTel.
> >>>>>
> >>>>> What about adding request IDs to events [1][2]? What's the intended use
> >>>>> case for that? Could you share some context here too?
> >>>>>
> >>>>> Side note: I proposed [2877] flagging event persistence as "beta" in
> >>>>> 1.2.0... This discussion adds another point towards that, I think.
> >>>>>
> >>>>> [1]
> >>>>>
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
> >>>>> [2]
> >>>>>
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
> >>>>>
> >>>>> [2877]
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
> >>>>>
> >>>>> Thanks,
> >>>>> Dmitri.
> >>>>>
> >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> >>> collado.mike@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey Dmitri
> >>>>>>
> >>>>>> The generating a request id is new code that was added after the
> >>> original
> >>>>>> x-request-id support. You can see the state from ~1 year ago, we
> >>> hard-coded
> >>>>>> request_id as the header we used for the MDC -
> >>>>>>
> >>>>>>
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
> >>>>>> . At some point, it was changed to be configurable, then the
> >>>>>> ContextResolverFilter filter was refactored/eliminated and the
> >>>>>> RequestIdFilter took its responsibility, but lost some of its original
> >>>>>> functionality.
> >>>>>>
> >>>>>> My personal opinion is that restoring support for the x-request-id
> >>> header
> >>>>>> is something that we should do, but if the header isn't present,
> >>> falling
> >>>>>> back on simply using OTel trace ids is good enough (better, even) than
> >>>>>> generating another random request id.
> >>>>>>
> >>>>>> Mike
> >>>>>>
> >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> >>> dimas@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Michael,
> >>>>>>>
> >>>>>>> Thanks for the info!
> >>>>>>>
> >>>>>>> Working with Envoy's tracing headers makes sense to me. However, I
> >>>>>> wonder:
> >>>>>>> why would Polaris need to generate a new request ID inside its
> >>> code?..
> >>>>>>> and return it in response headers?
> >>>>>>>
> >>>>>>> How important is it to propagate this ID to Polaris Events?
> >>>>>>>
> >>>>>>> I'm just trying to understand the full context :)
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dmitri.
> >>>>>>>
> >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> >>> collado.mike@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I think the original intention for this requestId field was to
> >>> support
> >>>>>>>> request propagation from load balancers, like Envoy (
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
> >>>>>>>> ), which is distinct from OTEL. Don't ask me why the default
> >>>>>>>> is Polaris-Request-Id - I think it was originally a custom thing,
> >>> but
> >>>>>>> then
> >>>>>>>> we changed to integrate with existing conventions. Unfortunately,
> >>>>>> looking
> >>>>>>>> through the code, I think that the actual functional plumbing has
> >>> been
> >>>>>>> lost
> >>>>>>>> in the course of multiple refactors around the call context and
> >>>>>>> resolver. I
> >>>>>>>> don't see references to that property or the underlying header.
> >>>>>>>>
> >>>>>>>> Support for the unofficial x-request-id header feels like something
> >>> we
> >>>>>>>> should definitely keep, especially when Polaris is one service in a
> >>>>>> mesh
> >>>>>>> of
> >>>>>>>> services that maybe don't have OTel integration. I'm a fan of the
> >>> OTel
> >>>>>>>> standard, but it's not entirely ubiquitous and there are many
> >>>>>> middleware
> >>>>>>>> layers that know how to forward on the x-request-id header.
> >>>>>>>>
> >>>>>>>> Mike
> >>>>>>>>
> >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> Yes, we should aim for interoperability with the existing de-facto
> >>>>>>>>> standard OTel and make it easy for users to integrate into their
> >>>>>>>>> observability platforms.
> >>>>>>>>>
> >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> >>>>>> dimas@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi Alex and All,
> >>>>>>>>>>
> >>>>>>>>>> I certainly support the idea of following OTel standards for
> >>>>>>> achieving
> >>>>>>>>>> "correlation" wrt Polaris requests and/or events.
> >>>>>>>>>>
> >>>>>>>>>> As to what form the correlation data should take, I believe it is
> >>>>>>>>>> conceptually what the OTel "context" represents. So, I believe it
> >>>>>>> makes
> >>>>>>>>>> sense for Polaris to support standard context propagators at the
> >>>>>> API
> >>>>>>>>> layer.
> >>>>>>>>>>
> >>>>>>>>>> If the incoming request has OTel context information, then
> >>>>>> returning
> >>>>>>>> any
> >>>>>>>>>> other "correlation" data in the response is redundant, I think.
> >>>>>>>>>>
> >>>>>>>>>> If the incoming request does not have OTel context info, what is
> >>>>>> the
> >>>>>>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
> >>>>>> it
> >>>>>>>>>> envisioned to be used?
> >>>>>>>>>>
> >>>>>>>>>> If the intention is to correlate a Polaris response (operation)
> >>>>>> with
> >>>>>>>>> events
> >>>>>>>>>> that might have resulted from its execution, I believe a more
> >>>>>> robust
> >>>>>>>>>> approach would be to propagate the OTel trace info (starting a new
> >>>>>>>> trace
> >>>>>>>>> if
> >>>>>>>>>> necessary) into event data. Then, Polaris can also return the
> >>> trace
> >>>>>>>>> context
> >>>>>>>>>> in the API response (top span). It's a bit awkward from the OTel
> >>>>>>>>>> perspective, but might be an option for supporting custom
> >>>>>>> correlators.
> >>>>>>>> It
> >>>>>>>>>> could be covered by a feature flag. The header name could be
> >>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
> >>>>>>>>>>
> >>>>>>>>>> Custom correlation code will be able to extract the trace ID from
> >>>>>> the
> >>>>>>>>>> response and from events and find related data. Granted, it will
> >>>>>>>> require
> >>>>>>>>> a
> >>>>>>>>>> bit more effort for the custom code to decode trace IDs from the
> >>>>>> OTel
> >>>>>>>>>> context, but the format is standard and not complex. The benefit
> >>>>>> for
> >>>>>>>>>> Polaris, though, is that it can easily integrate with
> >>>>>> OTel-compatible
> >>>>>>>>>> observability platforms regardless of whether any particular
> >>>>>>> deployment
> >>>>>>>>>> uses custom correlators or not.
> >>>>>>>>>>
> >>>>>>>>>> WDYT?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Dmitri.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> >>> adutra@apache.org
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi all,
> >>>>>>>>>>>
> >>>>>>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
> >>>>>>> not
> >>>>>>>>>>> entirely clear. It seems to serve as an observability feature to
> >>>>>>>>>>> facilitate correlation. A pending PR aims to rename it to
> >>>>>>>> "correlation
> >>>>>>>>>>> ID" for better alignment with industry standards [1].
> >>>>>>>>>>>
> >>>>>>>>>>> However, this PR has brought to light overlaps with core
> >>>>>> telemetry
> >>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> >>>>>>>>>>> request already has a trace ID and span ID, making a separate
> >>>>>>>>>>> correlation ID redundant.
> >>>>>>>>>>>
> >>>>>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
> >>>>>>>>>>> rather than the generated correlation ID, would significantly
> >>>>>>>> simplify
> >>>>>>>>>>> correlation of events with other traces.
> >>>>>>>>>>>
> >>>>>>>>>>> Therefore, I propose the following changes:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> >>>>>>>> correlation
> >>>>>>>>>>> ID for the request, instead of generating a random correlation
> >>>>>> ID.
> >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
> >>>>>>>> present
> >>>>>>>>>>> in the request, use it.
> >>>>>>>>>>> 3. If neither of the above conditions is met, generate a random
> >>>>>>>>>>> correlation ID.
> >>>>>>>>>>>
> >>>>>>>>>>> I am somewhat undecided on the best approach when a correlation
> >>>>>> ID
> >>>>>>>>>>> header is present in the request. However, I believe it would be
> >>>>>>> more
> >>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
> >>>>>> more
> >>>>>>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
> >>>>>>>> Trace
> >>>>>>>>>>> Context propagation [2].
> >>>>>>>>>>>
> >>>>>>>>>>> Please share your thoughts!
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Alex
> >>>>>>>>>>>
> >>>>>>>>>>> [1]:
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
> >>>>>>>>>>> [2]:
> >>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi all,
+1 to all of Alex’s AIs with Dmitri’s suggested changes as well. Great find, Dmitri!
I’m still debating with myself on whether we need to store the `x-request-id` field as part of the Events persistence. Can we think of a good use case where this is more helpful to the user than the OTel Trace/Span IDs? I am making the assumption here that those are still being returned back to the client.
Best,
Adnan Hemani
> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra <ad...@apache.org> wrote:
>
> Hi Dmitri,
>
> Yes, I think your suggestion to use just one field w3c_trace_context
> makes more sense than two fields (span_id and trace_id).
>
> With that, I also think that we are slowly drifting into
> implementation considerations; let's get consensus on the general
> design first, and we can certainly fine-tune the actual Java methods
> and SQL table columns in the future PR. WDYT?
>
> Thanks,
> Alex
>
> On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>>
>> Hi All,
>>
>> Many thanks for the background info, Adnan!
>>
>> +1 to action items proposed by Alex!
>>
>> Re: (3) Can we abstract request/trace info into a separate object without
>> exposing those accessors on the Event class directly? OTel defines
>> trace/span concepts, but in request ID is a bit foreign to OTel. Having
>> tracing / request ID isolated in java could help with maintaining it and
>> potentially supporting other (custom) tracing methods.
>>
>> Re: (4) I'd like to propose storing OTel correlation data in the form of a
>> standard context propagation string, e.g. W3C trace-context [1] (same value
>> as its HTTP header), so the column could be called w3c_trace_context or
>> simply trace_context.
>>
>> Open question: do we need to write a separate, individual trace ID field in
>> SQL? I suppose it is not very useful since correlating it to other trace
>> data already requires understanding OTel context propagation and a query
>> against trace_context can still be made using string-matching clauses. We
>> could probably (additionally) store it in the request_id column if the
>> Polaris-specific request ID header is not set.
>>
>> As for span ID, I do not really see a use case for persisting it
>> individually. It is very specific to OTel trace data construction.
>>
>> Actually, using the W3C trace context [1] encoding probably makes sense in
>> the java event representation too. Interested callers can easily decode
>> this information since the format is well-defined. As a side benefit, this
>> opens opportunities for downstream event consumers to connect (propagate
>> context) to traces that produced events based on the event data itself,
>> without relying on the intermediate frameworks. This may be desirable since
>> the current Polaris event persistence impl. writes events in batches, so
>> the association to individual requests that produced those events is lost.
>> Whether to perform this kind of context propagation will be at the
>> discretion of the event consumer, of course (outside of Polaris code).
>>
>> [1] https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK
>>
>> WDYT?
>>
>> Thanks,
>> Dmitri.
>>
>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org> wrote:
>>
>>> Hi all,
>>>
>>> Thank you for chiming in; the context around request IDs is now clear.
>>>
>>> Trying to summarize this thread into actionable items, here's what I
>>> propose:
>>>
>>> 1. Restore the original functionality for request IDs.
>>> - Change the default header name back to x-request-id (despite the
>>> x- prefix being deprecated), but keep it configurable as today.
>>> 2. Remove RequestIdGenerator and related functionality.
>>> - Do not generate a request ID if the REST client doesn't provide one.
>>> 3. Update PolarisEvent:
>>> - Expose new requestId(), traceId(), spanId() methods, all nullable.
>>> - This would align with the emerging consensus around including
>>> contextual information in PolarisEvent [1].
>>> 4. Update events table SQL schema:
>>> - Insert the client-provided request ID into the request_id
>>> column; otherwise, insert null.
>>> - Add two new nullable columns, trace_id and span_id, and populate
>>> them if OTel is enabled.
>>>
>>> From our discussions, I think it's important to not conflate OTel
>>> tracing fields with Envoy tracing fields, which is why I suggest we
>>> use separate fields / columns for them.
>>>
>>> Would the above plan work for everyone?
>>>
>>> Thanks,
>>> Alex
>>>
>>> [1]: https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T
>>>
>>>
>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
>>> <ad...@snowflake.com.invalid> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Thanks to Alex for starting this thread - because of this, I’m just
>>> coming to the realization that OTel Trace and Span IDs are coming built-in
>>> with Quarkus and my previous work to generate a Request/Correlation ID is
>>> likely not needed as a result. My original motivation for generation of a
>>> Request/Correlation ID was to ensure that any client can uniquely identify
>>> a request made to Polaris, which would be especially useful for debugging
>>> failing requests or identifying call patterns.
>>>>
>>>> As a result, I’m a +1 on Michael’s opinion: we should remove the
>>> Request/Correlation ID generation and always use the OTel trace/span IDs
>>> (which come for free with Quarkus) instead for the Correlation ID unless a
>>> valid header is present, which would take over as the Correlation ID
>>> instead.
>>>>
>>>> —
>>>>
>>>> To answer Dmitri’s question re Polaris Events: The intended use case is
>>> to provide some sort of correlation between events that have occurred as
>>> part of the same request. For example, if a user makes an CommitTransaction
>>> request, it would be helpful to see all UpdateTable calls that were made as
>>> part of that one user request.
>>>>
>>>> Best,
>>>> Adnan Hemani
>>>>
>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org>
>>> wrote:
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> Logging x-request-id headers makes sense.
>>>>>
>>>>> Just to confirm: if / when we restore that, Polaris will NOT generate
>>> new
>>>>> IDs in case the header is not present in the request, correct?
>>>>>
>>>>> I believe x-request-id can co-exist with OTel.
>>>>>
>>>>> What about adding request IDs to events [1][2]? What's the intended use
>>>>> case for that? Could you share some context here too?
>>>>>
>>>>> Side note: I proposed [2877] flagging event persistence as "beta" in
>>>>> 1.2.0... This discussion adds another point towards that, I think.
>>>>>
>>>>> [1]
>>>>>
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH
>>>>> [2]
>>>>>
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm
>>>>>
>>>>> [2877]
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H
>>>>>
>>>>> Thanks,
>>>>> Dmitri.
>>>>>
>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
>>> collado.mike@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hey Dmitri
>>>>>>
>>>>>> The generating a request id is new code that was added after the
>>> original
>>>>>> x-request-id support. You can see the state from ~1 year ago, we
>>> hard-coded
>>>>>> request_id as the header we used for the MDC -
>>>>>>
>>>>>>
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6
>>>>>> . At some point, it was changed to be configurable, then the
>>>>>> ContextResolverFilter filter was refactored/eliminated and the
>>>>>> RequestIdFilter took its responsibility, but lost some of its original
>>>>>> functionality.
>>>>>>
>>>>>> My personal opinion is that restoring support for the x-request-id
>>> header
>>>>>> is something that we should do, but if the header isn't present,
>>> falling
>>>>>> back on simply using OTel trace ids is good enough (better, even) than
>>>>>> generating another random request id.
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
>>> dimas@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> Thanks for the info!
>>>>>>>
>>>>>>> Working with Envoy's tracing headers makes sense to me. However, I
>>>>>> wonder:
>>>>>>> why would Polaris need to generate a new request ID inside its
>>> code?..
>>>>>>> and return it in response headers?
>>>>>>>
>>>>>>> How important is it to propagate this ID to Polaris Events?
>>>>>>>
>>>>>>> I'm just trying to understand the full context :)
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dmitri.
>>>>>>>
>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
>>> collado.mike@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I think the original intention for this requestId field was to
>>> support
>>>>>>>> request propagation from load balancers, like Envoy (
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa
>>>>>>>> ), which is distinct from OTEL. Don't ask me why the default
>>>>>>>> is Polaris-Request-Id - I think it was originally a custom thing,
>>> but
>>>>>>> then
>>>>>>>> we changed to integrate with existing conventions. Unfortunately,
>>>>>> looking
>>>>>>>> through the code, I think that the actual functional plumbing has
>>> been
>>>>>>> lost
>>>>>>>> in the course of multiple refactors around the call context and
>>>>>>> resolver. I
>>>>>>>> don't see references to that property or the underlying header.
>>>>>>>>
>>>>>>>> Support for the unofficial x-request-id header feels like something
>>> we
>>>>>>>> should definitely keep, especially when Polaris is one service in a
>>>>>> mesh
>>>>>>> of
>>>>>>>> services that maybe don't have OTel integration. I'm a fan of the
>>> OTel
>>>>>>>> standard, but it's not entirely ubiquitous and there are many
>>>>>> middleware
>>>>>>>> layers that know how to forward on the x-request-id header.
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
>>> wrote:
>>>>>>>>
>>>>>>>>> Yes, we should aim for interoperability with the existing de-facto
>>>>>>>>> standard OTel and make it easy for users to integrate into their
>>>>>>>>> observability platforms.
>>>>>>>>>
>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
>>>>>> dimas@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Alex and All,
>>>>>>>>>>
>>>>>>>>>> I certainly support the idea of following OTel standards for
>>>>>>> achieving
>>>>>>>>>> "correlation" wrt Polaris requests and/or events.
>>>>>>>>>>
>>>>>>>>>> As to what form the correlation data should take, I believe it is
>>>>>>>>>> conceptually what the OTel "context" represents. So, I believe it
>>>>>>> makes
>>>>>>>>>> sense for Polaris to support standard context propagators at the
>>>>>> API
>>>>>>>>> layer.
>>>>>>>>>>
>>>>>>>>>> If the incoming request has OTel context information, then
>>>>>> returning
>>>>>>>> any
>>>>>>>>>> other "correlation" data in the response is redundant, I think.
>>>>>>>>>>
>>>>>>>>>> If the incoming request does not have OTel context info, what is
>>>>>> the
>>>>>>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
>>>>>> it
>>>>>>>>>> envisioned to be used?
>>>>>>>>>>
>>>>>>>>>> If the intention is to correlate a Polaris response (operation)
>>>>>> with
>>>>>>>>> events
>>>>>>>>>> that might have resulted from its execution, I believe a more
>>>>>> robust
>>>>>>>>>> approach would be to propagate the OTel trace info (starting a new
>>>>>>>> trace
>>>>>>>>> if
>>>>>>>>>> necessary) into event data. Then, Polaris can also return the
>>> trace
>>>>>>>>> context
>>>>>>>>>> in the API response (top span). It's a bit awkward from the OTel
>>>>>>>>>> perspective, but might be an option for supporting custom
>>>>>>> correlators.
>>>>>>>> It
>>>>>>>>>> could be covered by a feature flag. The header name could be
>>>>>>>>>> "polaris-traceparent" for W3C Trace Context.
>>>>>>>>>>
>>>>>>>>>> Custom correlation code will be able to extract the trace ID from
>>>>>> the
>>>>>>>>>> response and from events and find related data. Granted, it will
>>>>>>>> require
>>>>>>>>> a
>>>>>>>>>> bit more effort for the custom code to decode trace IDs from the
>>>>>> OTel
>>>>>>>>>> context, but the format is standard and not complex. The benefit
>>>>>> for
>>>>>>>>>> Polaris, though, is that it can easily integrate with
>>>>>> OTel-compatible
>>>>>>>>>> observability platforms regardless of whether any particular
>>>>>>> deployment
>>>>>>>>>> uses custom correlators or not.
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Dmitri.
>>>>>>>>>>
>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
>>> adutra@apache.org
>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
>>>>>>> not
>>>>>>>>>>> entirely clear. It seems to serve as an observability feature to
>>>>>>>>>>> facilitate correlation. A pending PR aims to rename it to
>>>>>>>> "correlation
>>>>>>>>>>> ID" for better alignment with industry standards [1].
>>>>>>>>>>>
>>>>>>>>>>> However, this PR has brought to light overlaps with core
>>>>>> telemetry
>>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
>>>>>>>>>>> request already has a trace ID and span ID, making a separate
>>>>>>>>>>> correlation ID redundant.
>>>>>>>>>>>
>>>>>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
>>>>>>>>>>> rather than the generated correlation ID, would significantly
>>>>>>>> simplify
>>>>>>>>>>> correlation of events with other traces.
>>>>>>>>>>>
>>>>>>>>>>> Therefore, I propose the following changes:
>>>>>>>>>>>
>>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
>>>>>>>> correlation
>>>>>>>>>>> ID for the request, instead of generating a random correlation
>>>>>> ID.
>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
>>>>>>>> present
>>>>>>>>>>> in the request, use it.
>>>>>>>>>>> 3. If neither of the above conditions is met, generate a random
>>>>>>>>>>> correlation ID.
>>>>>>>>>>>
>>>>>>>>>>> I am somewhat undecided on the best approach when a correlation
>>>>>> ID
>>>>>>>>>>> header is present in the request. However, I believe it would be
>>>>>>> more
>>>>>>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
>>>>>> more
>>>>>>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
>>>>>>>> Trace
>>>>>>>>>>> Context propagation [2].
>>>>>>>>>>>
>>>>>>>>>>> Please share your thoughts!
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Alex
>>>>>>>>>>>
>>>>>>>>>>> [1]:
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg
>>>>>>>>>>> [2]:
>>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi Dmitri,
Yes, I think your suggestion to use just one field w3c_trace_context
makes more sense than two fields (span_id and trace_id).
With that, I also think that we are slowly drifting into
implementation considerations; let's get consensus on the general
design first, and we can certainly fine-tune the actual Java methods
and SQL table columns in the future PR. WDYT?
Thanks,
Alex
On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi All,
>
> Many thanks for the background info, Adnan!
>
> +1 to action items proposed by Alex!
>
> Re: (3) Can we abstract request/trace info into a separate object without
> exposing those accessors on the Event class directly? OTel defines
> trace/span concepts, but in request ID is a bit foreign to OTel. Having
> tracing / request ID isolated in java could help with maintaining it and
> potentially supporting other (custom) tracing methods.
>
> Re: (4) I'd like to propose storing OTel correlation data in the form of a
> standard context propagation string, e.g. W3C trace-context [1] (same value
> as its HTTP header), so the column could be called w3c_trace_context or
> simply trace_context.
>
> Open question: do we need to write a separate, individual trace ID field in
> SQL? I suppose it is not very useful since correlating it to other trace
> data already requires understanding OTel context propagation and a query
> against trace_context can still be made using string-matching clauses. We
> could probably (additionally) store it in the request_id column if the
> Polaris-specific request ID header is not set.
>
> As for span ID, I do not really see a use case for persisting it
> individually. It is very specific to OTel trace data construction.
>
> Actually, using the W3C trace context [1] encoding probably makes sense in
> the java event representation too. Interested callers can easily decode
> this information since the format is well-defined. As a side benefit, this
> opens opportunities for downstream event consumers to connect (propagate
> context) to traces that produced events based on the event data itself,
> without relying on the intermediate frameworks. This may be desirable since
> the current Polaris event persistence impl. writes events in batches, so
> the association to individual requests that produced those events is lost.
> Whether to perform this kind of context propagation will be at the
> discretion of the event consumer, of course (outside of Polaris code).
>
> [1] https://www.w3.org/TR/trace-context/
>
> WDYT?
>
> Thanks,
> Dmitri.
>
> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hi all,
> >
> > Thank you for chiming in; the context around request IDs is now clear.
> >
> > Trying to summarize this thread into actionable items, here's what I
> > propose:
> >
> > 1. Restore the original functionality for request IDs.
> > - Change the default header name back to x-request-id (despite the
> > x- prefix being deprecated), but keep it configurable as today.
> > 2. Remove RequestIdGenerator and related functionality.
> > - Do not generate a request ID if the REST client doesn't provide one.
> > 3. Update PolarisEvent:
> > - Expose new requestId(), traceId(), spanId() methods, all nullable.
> > - This would align with the emerging consensus around including
> > contextual information in PolarisEvent [1].
> > 4. Update events table SQL schema:
> > - Insert the client-provided request ID into the request_id
> > column; otherwise, insert null.
> > - Add two new nullable columns, trace_id and span_id, and populate
> > them if OTel is enabled.
> >
> > From our discussions, I think it's important to not conflate OTel
> > tracing fields with Envoy tracing fields, which is why I suggest we
> > use separate fields / columns for them.
> >
> > Would the above plan work for everyone?
> >
> > Thanks,
> > Alex
> >
> > [1]: https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy
> >
> >
> > On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> > <ad...@snowflake.com.invalid> wrote:
> > >
> > > Hi all,
> > >
> > > Thanks to Alex for starting this thread - because of this, I’m just
> > coming to the realization that OTel Trace and Span IDs are coming built-in
> > with Quarkus and my previous work to generate a Request/Correlation ID is
> > likely not needed as a result. My original motivation for generation of a
> > Request/Correlation ID was to ensure that any client can uniquely identify
> > a request made to Polaris, which would be especially useful for debugging
> > failing requests or identifying call patterns.
> > >
> > > As a result, I’m a +1 on Michael’s opinion: we should remove the
> > Request/Correlation ID generation and always use the OTel trace/span IDs
> > (which come for free with Quarkus) instead for the Correlation ID unless a
> > valid header is present, which would take over as the Correlation ID
> > instead.
> > >
> > > —
> > >
> > > To answer Dmitri’s question re Polaris Events: The intended use case is
> > to provide some sort of correlation between events that have occurred as
> > part of the same request. For example, if a user makes an CommitTransaction
> > request, it would be helpful to see all UpdateTable calls that were made as
> > part of that one user request.
> > >
> > > Best,
> > > Adnan Hemani
> > >
> > > > On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> > > >
> > > > Hi Michael,
> > > >
> > > > Logging x-request-id headers makes sense.
> > > >
> > > > Just to confirm: if / when we restore that, Polaris will NOT generate
> > new
> > > > IDs in case the header is not present in the request, correct?
> > > >
> > > > I believe x-request-id can co-exist with OTel.
> > > >
> > > > What about adding request IDs to events [1][2]? What's the intended use
> > > > case for that? Could you share some context here too?
> > > >
> > > > Side note: I proposed [2877] flagging event persistence as "beta" in
> > > > 1.2.0... This discussion adds another point towards that, I think.
> > > >
> > > > [1]
> > > >
> > https://www.google.com/url?q=https://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%23L97&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1WfUaXLp6z_M87iAXEqSUw
> > > > [2]
> > > >
> > https://www.google.com/url?q=https://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%23L66&source=gmail-imap&ust=1761851831000000&usg=AOvVaw12-7e3ahm2sLSkLSNqTecm
> > > >
> > > > [2877]
> > https://www.google.com/url?q=https://github.com/apache/polaris/pull/2877%23discussion_r2456300613&source=gmail-imap&ust=1761851831000000&usg=AOvVaw3TuYbkzwnLx3QEVIM8oDda
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> > > > On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> > collado.mike@gmail.com>
> > > > wrote:
> > > >
> > > >> Hey Dmitri
> > > >>
> > > >> The generating a request id is new code that was added after the
> > original
> > > >> x-request-id support. You can see the state from ~1 year ago, we
> > hard-coded
> > > >> request_id as the header we used for the MDC -
> > > >>
> > > >>
> > https://www.google.com/url?q=https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%23L415-L416&source=gmail-imap&ust=1761851831000000&usg=AOvVaw35Q1A_2avAiSYlYVAnZxBb
> > > >> . At some point, it was changed to be configurable, then the
> > > >> ContextResolverFilter filter was refactored/eliminated and the
> > > >> RequestIdFilter took its responsibility, but lost some of its original
> > > >> functionality.
> > > >>
> > > >> My personal opinion is that restoring support for the x-request-id
> > header
> > > >> is something that we should do, but if the header isn't present,
> > falling
> > > >> back on simply using OTel trace ids is good enough (better, even) than
> > > >> generating another random request id.
> > > >>
> > > >> Mike
> > > >>
> > > >> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> > dimas@apache.org>
> > > >> wrote:
> > > >>
> > > >>> Hi Michael,
> > > >>>
> > > >>> Thanks for the info!
> > > >>>
> > > >>> Working with Envoy's tracing headers makes sense to me. However, I
> > > >> wonder:
> > > >>> why would Polaris need to generate a new request ID inside its
> > code?..
> > > >>> and return it in response headers?
> > > >>>
> > > >>> How important is it to propagate this ID to Polaris Events?
> > > >>>
> > > >>> I'm just trying to understand the full context :)
> > > >>>
> > > >>> Thanks,
> > > >>> Dmitri.
> > > >>>
> > > >>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> > collado.mike@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> I think the original intention for this requestId field was to
> > support
> > > >>>> request propagation from load balancers, like Envoy (
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > https://www.google.com/url?q=https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1RnuM8mViV-j7jvuxq74Aw
> > > >>>> ), which is distinct from OTEL. Don't ask me why the default
> > > >>>> is Polaris-Request-Id - I think it was originally a custom thing,
> > but
> > > >>> then
> > > >>>> we changed to integrate with existing conventions. Unfortunately,
> > > >> looking
> > > >>>> through the code, I think that the actual functional plumbing has
> > been
> > > >>> lost
> > > >>>> in the course of multiple refactors around the call context and
> > > >>> resolver. I
> > > >>>> don't see references to that property or the underlying header.
> > > >>>>
> > > >>>> Support for the unofficial x-request-id header feels like something
> > we
> > > >>>> should definitely keep, especially when Polaris is one service in a
> > > >> mesh
> > > >>> of
> > > >>>> services that maybe don't have OTel integration. I'm a fan of the
> > OTel
> > > >>>> standard, but it's not entirely ubiquitous and there are many
> > > >> middleware
> > > >>>> layers that know how to forward on the x-request-id header.
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> > wrote:
> > > >>>>
> > > >>>>> Yes, we should aim for interoperability with the existing de-facto
> > > >>>>> standard OTel and make it easy for users to integrate into their
> > > >>>>> observability platforms.
> > > >>>>>
> > > >>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > > >> dimas@apache.org>
> > > >>>>> wrote:
> > > >>>>>>
> > > >>>>>> Hi Alex and All,
> > > >>>>>>
> > > >>>>>> I certainly support the idea of following OTel standards for
> > > >>> achieving
> > > >>>>>> "correlation" wrt Polaris requests and/or events.
> > > >>>>>>
> > > >>>>>> As to what form the correlation data should take, I believe it is
> > > >>>>>> conceptually what the OTel "context" represents. So, I believe it
> > > >>> makes
> > > >>>>>> sense for Polaris to support standard context propagators at the
> > > >> API
> > > >>>>> layer.
> > > >>>>>>
> > > >>>>>> If the incoming request has OTel context information, then
> > > >> returning
> > > >>>> any
> > > >>>>>> other "correlation" data in the response is redundant, I think.
> > > >>>>>>
> > > >>>>>> If the incoming request does not have OTel context info, what is
> > > >> the
> > > >>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
> > > >> it
> > > >>>>>> envisioned to be used?
> > > >>>>>>
> > > >>>>>> If the intention is to correlate a Polaris response (operation)
> > > >> with
> > > >>>>> events
> > > >>>>>> that might have resulted from its execution, I believe a more
> > > >> robust
> > > >>>>>> approach would be to propagate the OTel trace info (starting a new
> > > >>>> trace
> > > >>>>> if
> > > >>>>>> necessary) into event data. Then, Polaris can also return the
> > trace
> > > >>>>> context
> > > >>>>>> in the API response (top span). It's a bit awkward from the OTel
> > > >>>>>> perspective, but might be an option for supporting custom
> > > >>> correlators.
> > > >>>> It
> > > >>>>>> could be covered by a feature flag. The header name could be
> > > >>>>>> "polaris-traceparent" for W3C Trace Context.
> > > >>>>>>
> > > >>>>>> Custom correlation code will be able to extract the trace ID from
> > > >> the
> > > >>>>>> response and from events and find related data. Granted, it will
> > > >>>> require
> > > >>>>> a
> > > >>>>>> bit more effort for the custom code to decode trace IDs from the
> > > >> OTel
> > > >>>>>> context, but the format is standard and not complex. The benefit
> > > >> for
> > > >>>>>> Polaris, though, is that it can easily integrate with
> > > >> OTel-compatible
> > > >>>>>> observability platforms regardless of whether any particular
> > > >>> deployment
> > > >>>>>> uses custom correlators or not.
> > > >>>>>>
> > > >>>>>> WDYT?
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Dmitri.
> > > >>>>>>
> > > >>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> > adutra@apache.org
> > > >>>
> > > >>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi all,
> > > >>>>>>>
> > > >>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
> > > >>> not
> > > >>>>>>> entirely clear. It seems to serve as an observability feature to
> > > >>>>>>> facilitate correlation. A pending PR aims to rename it to
> > > >>>> "correlation
> > > >>>>>>> ID" for better alignment with industry standards [1].
> > > >>>>>>>
> > > >>>>>>> However, this PR has brought to light overlaps with core
> > > >> telemetry
> > > >>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > > >>>>>>> request already has a trace ID and span ID, making a separate
> > > >>>>>>> correlation ID redundant.
> > > >>>>>>>
> > > >>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
> > > >>>>>>> rather than the generated correlation ID, would significantly
> > > >>>> simplify
> > > >>>>>>> correlation of events with other traces.
> > > >>>>>>>
> > > >>>>>>> Therefore, I propose the following changes:
> > > >>>>>>>
> > > >>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> > > >>>> correlation
> > > >>>>>>> ID for the request, instead of generating a random correlation
> > > >> ID.
> > > >>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
> > > >>>> present
> > > >>>>>>> in the request, use it.
> > > >>>>>>> 3. If neither of the above conditions is met, generate a random
> > > >>>>>>> correlation ID.
> > > >>>>>>>
> > > >>>>>>> I am somewhat undecided on the best approach when a correlation
> > > >> ID
> > > >>>>>>> header is present in the request. However, I believe it would be
> > > >>> more
> > > >>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
> > > >> more
> > > >>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
> > > >>>> Trace
> > > >>>>>>> Context propagation [2].
> > > >>>>>>>
> > > >>>>>>> Please share your thoughts!
> > > >>>>>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Alex
> > > >>>>>>>
> > > >>>>>>> [1]:
> > https://www.google.com/url?q=https://github.com/apache/polaris/pull/2757&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1-kAWfEk4tmsEg0q0GZBCn
> > > >>>>>>> [2]:
> > https://www.google.com/url?q=https://www.w3.org/TR/trace-context&source=gmail-imap&ust=1761851831000000&usg=AOvVaw22nMyOS7pbJ69XrBo5kHQS
> > > >>>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi All,
Many thanks for the background info, Adnan!
+1 to action items proposed by Alex!
Re: (3) Can we abstract request/trace info into a separate object without
exposing those accessors on the Event class directly? OTel defines
trace/span concepts, but in request ID is a bit foreign to OTel. Having
tracing / request ID isolated in java could help with maintaining it and
potentially supporting other (custom) tracing methods.
Re: (4) I'd like to propose storing OTel correlation data in the form of a
standard context propagation string, e.g. W3C trace-context [1] (same value
as its HTTP header), so the column could be called w3c_trace_context or
simply trace_context.
Open question: do we need to write a separate, individual trace ID field in
SQL? I suppose it is not very useful since correlating it to other trace
data already requires understanding OTel context propagation and a query
against trace_context can still be made using string-matching clauses. We
could probably (additionally) store it in the request_id column if the
Polaris-specific request ID header is not set.
As for span ID, I do not really see a use case for persisting it
individually. It is very specific to OTel trace data construction.
Actually, using the W3C trace context [1] encoding probably makes sense in
the java event representation too. Interested callers can easily decode
this information since the format is well-defined. As a side benefit, this
opens opportunities for downstream event consumers to connect (propagate
context) to traces that produced events based on the event data itself,
without relying on the intermediate frameworks. This may be desirable since
the current Polaris event persistence impl. writes events in batches, so
the association to individual requests that produced those events is lost.
Whether to perform this kind of context propagation will be at the
discretion of the event consumer, of course (outside of Polaris code).
[1] https://www.w3.org/TR/trace-context/
WDYT?
Thanks,
Dmitri.
On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra <ad...@apache.org> wrote:
> Hi all,
>
> Thank you for chiming in; the context around request IDs is now clear.
>
> Trying to summarize this thread into actionable items, here's what I
> propose:
>
> 1. Restore the original functionality for request IDs.
> - Change the default header name back to x-request-id (despite the
> x- prefix being deprecated), but keep it configurable as today.
> 2. Remove RequestIdGenerator and related functionality.
> - Do not generate a request ID if the REST client doesn't provide one.
> 3. Update PolarisEvent:
> - Expose new requestId(), traceId(), spanId() methods, all nullable.
> - This would align with the emerging consensus around including
> contextual information in PolarisEvent [1].
> 4. Update events table SQL schema:
> - Insert the client-provided request ID into the request_id
> column; otherwise, insert null.
> - Add two new nullable columns, trace_id and span_id, and populate
> them if OTel is enabled.
>
> From our discussions, I think it's important to not conflate OTel
> tracing fields with Envoy tracing fields, which is why I suggest we
> use separate fields / columns for them.
>
> Would the above plan work for everyone?
>
> Thanks,
> Alex
>
> [1]: https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy
>
>
> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
> <ad...@snowflake.com.invalid> wrote:
> >
> > Hi all,
> >
> > Thanks to Alex for starting this thread - because of this, I’m just
> coming to the realization that OTel Trace and Span IDs are coming built-in
> with Quarkus and my previous work to generate a Request/Correlation ID is
> likely not needed as a result. My original motivation for generation of a
> Request/Correlation ID was to ensure that any client can uniquely identify
> a request made to Polaris, which would be especially useful for debugging
> failing requests or identifying call patterns.
> >
> > As a result, I’m a +1 on Michael’s opinion: we should remove the
> Request/Correlation ID generation and always use the OTel trace/span IDs
> (which come for free with Quarkus) instead for the Correlation ID unless a
> valid header is present, which would take over as the Correlation ID
> instead.
> >
> > —
> >
> > To answer Dmitri’s question re Polaris Events: The intended use case is
> to provide some sort of correlation between events that have occurred as
> part of the same request. For example, if a user makes an CommitTransaction
> request, it would be helpful to see all UpdateTable calls that were made as
> part of that one user request.
> >
> > Best,
> > Adnan Hemani
> >
> > > On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> > >
> > > Hi Michael,
> > >
> > > Logging x-request-id headers makes sense.
> > >
> > > Just to confirm: if / when we restore that, Polaris will NOT generate
> new
> > > IDs in case the header is not present in the request, correct?
> > >
> > > I believe x-request-id can co-exist with OTel.
> > >
> > > What about adding request IDs to events [1][2]? What's the intended use
> > > case for that? Could you share some context here too?
> > >
> > > Side note: I proposed [2877] flagging event persistence as "beta" in
> > > 1.2.0... This discussion adds another point towards that, I think.
> > >
> > > [1]
> > >
> https://www.google.com/url?q=https://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%23L97&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1WfUaXLp6z_M87iAXEqSUw
> > > [2]
> > >
> https://www.google.com/url?q=https://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%23L66&source=gmail-imap&ust=1761851831000000&usg=AOvVaw12-7e3ahm2sLSkLSNqTecm
> > >
> > > [2877]
> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2877%23discussion_r2456300613&source=gmail-imap&ust=1761851831000000&usg=AOvVaw3TuYbkzwnLx3QEVIM8oDda
> > >
> > > Thanks,
> > > Dmitri.
> > >
> > > On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <
> collado.mike@gmail.com>
> > > wrote:
> > >
> > >> Hey Dmitri
> > >>
> > >> The generating a request id is new code that was added after the
> original
> > >> x-request-id support. You can see the state from ~1 year ago, we
> hard-coded
> > >> request_id as the header we used for the MDC -
> > >>
> > >>
> https://www.google.com/url?q=https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%23L415-L416&source=gmail-imap&ust=1761851831000000&usg=AOvVaw35Q1A_2avAiSYlYVAnZxBb
> > >> . At some point, it was changed to be configurable, then the
> > >> ContextResolverFilter filter was refactored/eliminated and the
> > >> RequestIdFilter took its responsibility, but lost some of its original
> > >> functionality.
> > >>
> > >> My personal opinion is that restoring support for the x-request-id
> header
> > >> is something that we should do, but if the header isn't present,
> falling
> > >> back on simply using OTel trace ids is good enough (better, even) than
> > >> generating another random request id.
> > >>
> > >> Mike
> > >>
> > >> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <
> dimas@apache.org>
> > >> wrote:
> > >>
> > >>> Hi Michael,
> > >>>
> > >>> Thanks for the info!
> > >>>
> > >>> Working with Envoy's tracing headers makes sense to me. However, I
> > >> wonder:
> > >>> why would Polaris need to generate a new request ID inside its
> code?..
> > >>> and return it in response headers?
> > >>>
> > >>> How important is it to propagate this ID to Polaris Events?
> > >>>
> > >>> I'm just trying to understand the full context :)
> > >>>
> > >>> Thanks,
> > >>> Dmitri.
> > >>>
> > >>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <
> collado.mike@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> I think the original intention for this requestId field was to
> support
> > >>>> request propagation from load balancers, like Envoy (
> > >>>>
> > >>>>
> > >>>
> > >>
> https://www.google.com/url?q=https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1RnuM8mViV-j7jvuxq74Aw
> > >>>> ), which is distinct from OTEL. Don't ask me why the default
> > >>>> is Polaris-Request-Id - I think it was originally a custom thing,
> but
> > >>> then
> > >>>> we changed to integrate with existing conventions. Unfortunately,
> > >> looking
> > >>>> through the code, I think that the actual functional plumbing has
> been
> > >>> lost
> > >>>> in the course of multiple refactors around the call context and
> > >>> resolver. I
> > >>>> don't see references to that property or the underlying header.
> > >>>>
> > >>>> Support for the unofficial x-request-id header feels like something
> we
> > >>>> should definitely keep, especially when Polaris is one service in a
> > >> mesh
> > >>> of
> > >>>> services that maybe don't have OTel integration. I'm a fan of the
> OTel
> > >>>> standard, but it's not entirely ubiquitous and there are many
> > >> middleware
> > >>>> layers that know how to forward on the x-request-id header.
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de>
> wrote:
> > >>>>
> > >>>>> Yes, we should aim for interoperability with the existing de-facto
> > >>>>> standard OTel and make it easy for users to integrate into their
> > >>>>> observability platforms.
> > >>>>>
> > >>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> > >> dimas@apache.org>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> Hi Alex and All,
> > >>>>>>
> > >>>>>> I certainly support the idea of following OTel standards for
> > >>> achieving
> > >>>>>> "correlation" wrt Polaris requests and/or events.
> > >>>>>>
> > >>>>>> As to what form the correlation data should take, I believe it is
> > >>>>>> conceptually what the OTel "context" represents. So, I believe it
> > >>> makes
> > >>>>>> sense for Polaris to support standard context propagators at the
> > >> API
> > >>>>> layer.
> > >>>>>>
> > >>>>>> If the incoming request has OTel context information, then
> > >> returning
> > >>>> any
> > >>>>>> other "correlation" data in the response is redundant, I think.
> > >>>>>>
> > >>>>>> If the incoming request does not have OTel context info, what is
> > >> the
> > >>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
> > >> it
> > >>>>>> envisioned to be used?
> > >>>>>>
> > >>>>>> If the intention is to correlate a Polaris response (operation)
> > >> with
> > >>>>> events
> > >>>>>> that might have resulted from its execution, I believe a more
> > >> robust
> > >>>>>> approach would be to propagate the OTel trace info (starting a new
> > >>>> trace
> > >>>>> if
> > >>>>>> necessary) into event data. Then, Polaris can also return the
> trace
> > >>>>> context
> > >>>>>> in the API response (top span). It's a bit awkward from the OTel
> > >>>>>> perspective, but might be an option for supporting custom
> > >>> correlators.
> > >>>> It
> > >>>>>> could be covered by a feature flag. The header name could be
> > >>>>>> "polaris-traceparent" for W3C Trace Context.
> > >>>>>>
> > >>>>>> Custom correlation code will be able to extract the trace ID from
> > >> the
> > >>>>>> response and from events and find related data. Granted, it will
> > >>>> require
> > >>>>> a
> > >>>>>> bit more effort for the custom code to decode trace IDs from the
> > >> OTel
> > >>>>>> context, but the format is standard and not complex. The benefit
> > >> for
> > >>>>>> Polaris, though, is that it can easily integrate with
> > >> OTel-compatible
> > >>>>>> observability platforms regardless of whether any particular
> > >>> deployment
> > >>>>>> uses custom correlators or not.
> > >>>>>>
> > >>>>>> WDYT?
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Dmitri.
> > >>>>>>
> > >>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <
> adutra@apache.org
> > >>>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi all,
> > >>>>>>>
> > >>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
> > >>> not
> > >>>>>>> entirely clear. It seems to serve as an observability feature to
> > >>>>>>> facilitate correlation. A pending PR aims to rename it to
> > >>>> "correlation
> > >>>>>>> ID" for better alignment with industry standards [1].
> > >>>>>>>
> > >>>>>>> However, this PR has brought to light overlaps with core
> > >> telemetry
> > >>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > >>>>>>> request already has a trace ID and span ID, making a separate
> > >>>>>>> correlation ID redundant.
> > >>>>>>>
> > >>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
> > >>>>>>> rather than the generated correlation ID, would significantly
> > >>>> simplify
> > >>>>>>> correlation of events with other traces.
> > >>>>>>>
> > >>>>>>> Therefore, I propose the following changes:
> > >>>>>>>
> > >>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> > >>>> correlation
> > >>>>>>> ID for the request, instead of generating a random correlation
> > >> ID.
> > >>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
> > >>>> present
> > >>>>>>> in the request, use it.
> > >>>>>>> 3. If neither of the above conditions is met, generate a random
> > >>>>>>> correlation ID.
> > >>>>>>>
> > >>>>>>> I am somewhat undecided on the best approach when a correlation
> > >> ID
> > >>>>>>> header is present in the request. However, I believe it would be
> > >>> more
> > >>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
> > >> more
> > >>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
> > >>>> Trace
> > >>>>>>> Context propagation [2].
> > >>>>>>>
> > >>>>>>> Please share your thoughts!
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Alex
> > >>>>>>>
> > >>>>>>> [1]:
> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2757&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1-kAWfEk4tmsEg0q0GZBCn
> > >>>>>>> [2]:
> https://www.google.com/url?q=https://www.w3.org/TR/trace-context&source=gmail-imap&ust=1761851831000000&usg=AOvVaw22nMyOS7pbJ69XrBo5kHQS
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,
Thank you for chiming in; the context around request IDs is now clear.
Trying to summarize this thread into actionable items, here's what I propose:
1. Restore the original functionality for request IDs.
- Change the default header name back to x-request-id (despite the
x- prefix being deprecated), but keep it configurable as today.
2. Remove RequestIdGenerator and related functionality.
- Do not generate a request ID if the REST client doesn't provide one.
3. Update PolarisEvent:
- Expose new requestId(), traceId(), spanId() methods, all nullable.
- This would align with the emerging consensus around including
contextual information in PolarisEvent [1].
4. Update events table SQL schema:
- Insert the client-provided request ID into the request_id
column; otherwise, insert null.
- Add two new nullable columns, trace_id and span_id, and populate
them if OTel is enabled.
From our discussions, I think it's important to not conflate OTel
tracing fields with Envoy tracing fields, which is why I suggest we
use separate fields / columns for them.
Would the above plan work for everyone?
Thanks,
Alex
[1]: https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy
On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani
<ad...@snowflake.com.invalid> wrote:
>
> Hi all,
>
> Thanks to Alex for starting this thread - because of this, I’m just coming to the realization that OTel Trace and Span IDs are coming built-in with Quarkus and my previous work to generate a Request/Correlation ID is likely not needed as a result. My original motivation for generation of a Request/Correlation ID was to ensure that any client can uniquely identify a request made to Polaris, which would be especially useful for debugging failing requests or identifying call patterns.
>
> As a result, I’m a +1 on Michael’s opinion: we should remove the Request/Correlation ID generation and always use the OTel trace/span IDs (which come for free with Quarkus) instead for the Correlation ID unless a valid header is present, which would take over as the Correlation ID instead.
>
> —
>
> To answer Dmitri’s question re Polaris Events: The intended use case is to provide some sort of correlation between events that have occurred as part of the same request. For example, if a user makes an CommitTransaction request, it would be helpful to see all UpdateTable calls that were made as part of that one user request.
>
> Best,
> Adnan Hemani
>
> > On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org> wrote:
> >
> > Hi Michael,
> >
> > Logging x-request-id headers makes sense.
> >
> > Just to confirm: if / when we restore that, Polaris will NOT generate new
> > IDs in case the header is not present in the request, correct?
> >
> > I believe x-request-id can co-exist with OTel.
> >
> > What about adding request IDs to events [1][2]? What's the intended use
> > case for that? Could you share some context here too?
> >
> > Side note: I proposed [2877] flagging event persistence as "beta" in
> > 1.2.0... This discussion adds another point towards that, I think.
> >
> > [1]
> > https://www.google.com/url?q=https://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%23L97&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1WfUaXLp6z_M87iAXEqSUw
> > [2]
> > https://www.google.com/url?q=https://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%23L66&source=gmail-imap&ust=1761851831000000&usg=AOvVaw12-7e3ahm2sLSkLSNqTecm
> >
> > [2877] https://www.google.com/url?q=https://github.com/apache/polaris/pull/2877%23discussion_r2456300613&source=gmail-imap&ust=1761851831000000&usg=AOvVaw3TuYbkzwnLx3QEVIM8oDda
> >
> > Thanks,
> > Dmitri.
> >
> > On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <co...@gmail.com>
> > wrote:
> >
> >> Hey Dmitri
> >>
> >> The generating a request id is new code that was added after the original
> >> x-request-id support. You can see the state from ~1 year ago, we hard-coded
> >> request_id as the header we used for the MDC -
> >>
> >> https://www.google.com/url?q=https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%23L415-L416&source=gmail-imap&ust=1761851831000000&usg=AOvVaw35Q1A_2avAiSYlYVAnZxBb
> >> . At some point, it was changed to be configurable, then the
> >> ContextResolverFilter filter was refactored/eliminated and the
> >> RequestIdFilter took its responsibility, but lost some of its original
> >> functionality.
> >>
> >> My personal opinion is that restoring support for the x-request-id header
> >> is something that we should do, but if the header isn't present, falling
> >> back on simply using OTel trace ids is good enough (better, even) than
> >> generating another random request id.
> >>
> >> Mike
> >>
> >> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <di...@apache.org>
> >> wrote:
> >>
> >>> Hi Michael,
> >>>
> >>> Thanks for the info!
> >>>
> >>> Working with Envoy's tracing headers makes sense to me. However, I
> >> wonder:
> >>> why would Polaris need to generate a new request ID inside its code?..
> >>> and return it in response headers?
> >>>
> >>> How important is it to propagate this ID to Polaris Events?
> >>>
> >>> I'm just trying to understand the full context :)
> >>>
> >>> Thanks,
> >>> Dmitri.
> >>>
> >>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <co...@gmail.com>
> >>> wrote:
> >>>
> >>>> I think the original intention for this requestId field was to support
> >>>> request propagation from load balancers, like Envoy (
> >>>>
> >>>>
> >>>
> >> https://www.google.com/url?q=https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1RnuM8mViV-j7jvuxq74Aw
> >>>> ), which is distinct from OTEL. Don't ask me why the default
> >>>> is Polaris-Request-Id - I think it was originally a custom thing, but
> >>> then
> >>>> we changed to integrate with existing conventions. Unfortunately,
> >> looking
> >>>> through the code, I think that the actual functional plumbing has been
> >>> lost
> >>>> in the course of multiple refactors around the call context and
> >>> resolver. I
> >>>> don't see references to that property or the underlying header.
> >>>>
> >>>> Support for the unofficial x-request-id header feels like something we
> >>>> should definitely keep, especially when Polaris is one service in a
> >> mesh
> >>> of
> >>>> services that maybe don't have OTel integration. I'm a fan of the OTel
> >>>> standard, but it's not entirely ubiquitous and there are many
> >> middleware
> >>>> layers that know how to forward on the x-request-id header.
> >>>>
> >>>> Mike
> >>>>
> >>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
> >>>>
> >>>>> Yes, we should aim for interoperability with the existing de-facto
> >>>>> standard OTel and make it easy for users to integrate into their
> >>>>> observability platforms.
> >>>>>
> >>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> >> dimas@apache.org>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi Alex and All,
> >>>>>>
> >>>>>> I certainly support the idea of following OTel standards for
> >>> achieving
> >>>>>> "correlation" wrt Polaris requests and/or events.
> >>>>>>
> >>>>>> As to what form the correlation data should take, I believe it is
> >>>>>> conceptually what the OTel "context" represents. So, I believe it
> >>> makes
> >>>>>> sense for Polaris to support standard context propagators at the
> >> API
> >>>>> layer.
> >>>>>>
> >>>>>> If the incoming request has OTel context information, then
> >> returning
> >>>> any
> >>>>>> other "correlation" data in the response is redundant, I think.
> >>>>>>
> >>>>>> If the incoming request does not have OTel context info, what is
> >> the
> >>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
> >> it
> >>>>>> envisioned to be used?
> >>>>>>
> >>>>>> If the intention is to correlate a Polaris response (operation)
> >> with
> >>>>> events
> >>>>>> that might have resulted from its execution, I believe a more
> >> robust
> >>>>>> approach would be to propagate the OTel trace info (starting a new
> >>>> trace
> >>>>> if
> >>>>>> necessary) into event data. Then, Polaris can also return the trace
> >>>>> context
> >>>>>> in the API response (top span). It's a bit awkward from the OTel
> >>>>>> perspective, but might be an option for supporting custom
> >>> correlators.
> >>>> It
> >>>>>> could be covered by a feature flag. The header name could be
> >>>>>> "polaris-traceparent" for W3C Trace Context.
> >>>>>>
> >>>>>> Custom correlation code will be able to extract the trace ID from
> >> the
> >>>>>> response and from events and find related data. Granted, it will
> >>>> require
> >>>>> a
> >>>>>> bit more effort for the custom code to decode trace IDs from the
> >> OTel
> >>>>>> context, but the format is standard and not complex. The benefit
> >> for
> >>>>>> Polaris, though, is that it can easily integrate with
> >> OTel-compatible
> >>>>>> observability platforms regardless of whether any particular
> >>> deployment
> >>>>>> uses custom correlators or not.
> >>>>>>
> >>>>>> WDYT?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Dmitri.
> >>>>>>
> >>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <adutra@apache.org
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
> >>> not
> >>>>>>> entirely clear. It seems to serve as an observability feature to
> >>>>>>> facilitate correlation. A pending PR aims to rename it to
> >>>> "correlation
> >>>>>>> ID" for better alignment with industry standards [1].
> >>>>>>>
> >>>>>>> However, this PR has brought to light overlaps with core
> >> telemetry
> >>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> >>>>>>> request already has a trace ID and span ID, making a separate
> >>>>>>> correlation ID redundant.
> >>>>>>>
> >>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
> >>>>>>> rather than the generated correlation ID, would significantly
> >>>> simplify
> >>>>>>> correlation of events with other traces.
> >>>>>>>
> >>>>>>> Therefore, I propose the following changes:
> >>>>>>>
> >>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
> >>>> correlation
> >>>>>>> ID for the request, instead of generating a random correlation
> >> ID.
> >>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
> >>>> present
> >>>>>>> in the request, use it.
> >>>>>>> 3. If neither of the above conditions is met, generate a random
> >>>>>>> correlation ID.
> >>>>>>>
> >>>>>>> I am somewhat undecided on the best approach when a correlation
> >> ID
> >>>>>>> header is present in the request. However, I believe it would be
> >>> more
> >>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
> >> more
> >>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
> >>>> Trace
> >>>>>>> Context propagation [2].
> >>>>>>>
> >>>>>>> Please share your thoughts!
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Alex
> >>>>>>>
> >>>>>>> [1]: https://www.google.com/url?q=https://github.com/apache/polaris/pull/2757&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1-kAWfEk4tmsEg0q0GZBCn
> >>>>>>> [2]: https://www.google.com/url?q=https://www.w3.org/TR/trace-context&source=gmail-imap&ust=1761851831000000&usg=AOvVaw22nMyOS7pbJ69XrBo5kHQS
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Adnan Hemani <ad...@snowflake.com.INVALID>.
Hi all,
Thanks to Alex for starting this thread - because of this, I’m just coming to the realization that OTel Trace and Span IDs are coming built-in with Quarkus and my previous work to generate a Request/Correlation ID is likely not needed as a result. My original motivation for generation of a Request/Correlation ID was to ensure that any client can uniquely identify a request made to Polaris, which would be especially useful for debugging failing requests or identifying call patterns.
As a result, I’m a +1 on Michael’s opinion: we should remove the Request/Correlation ID generation and always use the OTel trace/span IDs (which come for free with Quarkus) instead for the Correlation ID unless a valid header is present, which would take over as the Correlation ID instead.
—
To answer Dmitri’s question re Polaris Events: The intended use case is to provide some sort of correlation between events that have occurred as part of the same request. For example, if a user makes an CommitTransaction request, it would be helpful to see all UpdateTable calls that were made as part of that one user request.
Best,
Adnan Hemani
> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Michael,
>
> Logging x-request-id headers makes sense.
>
> Just to confirm: if / when we restore that, Polaris will NOT generate new
> IDs in case the header is not present in the request, correct?
>
> I believe x-request-id can co-exist with OTel.
>
> What about adding request IDs to events [1][2]? What's the intended use
> case for that? Could you share some context here too?
>
> Side note: I proposed [2877] flagging event persistence as "beta" in
> 1.2.0... This discussion adds another point towards that, I think.
>
> [1]
> https://www.google.com/url?q=https://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%23L97&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1WfUaXLp6z_M87iAXEqSUw
> [2]
> https://www.google.com/url?q=https://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%23L66&source=gmail-imap&ust=1761851831000000&usg=AOvVaw12-7e3ahm2sLSkLSNqTecm
>
> [2877] https://www.google.com/url?q=https://github.com/apache/polaris/pull/2877%23discussion_r2456300613&source=gmail-imap&ust=1761851831000000&usg=AOvVaw3TuYbkzwnLx3QEVIM8oDda
>
> Thanks,
> Dmitri.
>
> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <co...@gmail.com>
> wrote:
>
>> Hey Dmitri
>>
>> The generating a request id is new code that was added after the original
>> x-request-id support. You can see the state from ~1 year ago, we hard-coded
>> request_id as the header we used for the MDC -
>>
>> https://www.google.com/url?q=https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%23L415-L416&source=gmail-imap&ust=1761851831000000&usg=AOvVaw35Q1A_2avAiSYlYVAnZxBb
>> . At some point, it was changed to be configurable, then the
>> ContextResolverFilter filter was refactored/eliminated and the
>> RequestIdFilter took its responsibility, but lost some of its original
>> functionality.
>>
>> My personal opinion is that restoring support for the x-request-id header
>> is something that we should do, but if the header isn't present, falling
>> back on simply using OTel trace ids is good enough (better, even) than
>> generating another random request id.
>>
>> Mike
>>
>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <di...@apache.org>
>> wrote:
>>
>>> Hi Michael,
>>>
>>> Thanks for the info!
>>>
>>> Working with Envoy's tracing headers makes sense to me. However, I
>> wonder:
>>> why would Polaris need to generate a new request ID inside its code?..
>>> and return it in response headers?
>>>
>>> How important is it to propagate this ID to Polaris Events?
>>>
>>> I'm just trying to understand the full context :)
>>>
>>> Thanks,
>>> Dmitri.
>>>
>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <co...@gmail.com>
>>> wrote:
>>>
>>>> I think the original intention for this requestId field was to support
>>>> request propagation from load balancers, like Envoy (
>>>>
>>>>
>>>
>> https://www.google.com/url?q=https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1RnuM8mViV-j7jvuxq74Aw
>>>> ), which is distinct from OTEL. Don't ask me why the default
>>>> is Polaris-Request-Id - I think it was originally a custom thing, but
>>> then
>>>> we changed to integrate with existing conventions. Unfortunately,
>> looking
>>>> through the code, I think that the actual functional plumbing has been
>>> lost
>>>> in the course of multiple refactors around the call context and
>>> resolver. I
>>>> don't see references to that property or the underlying header.
>>>>
>>>> Support for the unofficial x-request-id header feels like something we
>>>> should definitely keep, especially when Polaris is one service in a
>> mesh
>>> of
>>>> services that maybe don't have OTel integration. I'm a fan of the OTel
>>>> standard, but it's not entirely ubiquitous and there are many
>> middleware
>>>> layers that know how to forward on the x-request-id header.
>>>>
>>>> Mike
>>>>
>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
>>>>
>>>>> Yes, we should aim for interoperability with the existing de-facto
>>>>> standard OTel and make it easy for users to integrate into their
>>>>> observability platforms.
>>>>>
>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
>> dimas@apache.org>
>>>>> wrote:
>>>>>>
>>>>>> Hi Alex and All,
>>>>>>
>>>>>> I certainly support the idea of following OTel standards for
>>> achieving
>>>>>> "correlation" wrt Polaris requests and/or events.
>>>>>>
>>>>>> As to what form the correlation data should take, I believe it is
>>>>>> conceptually what the OTel "context" represents. So, I believe it
>>> makes
>>>>>> sense for Polaris to support standard context propagators at the
>> API
>>>>> layer.
>>>>>>
>>>>>> If the incoming request has OTel context information, then
>> returning
>>>> any
>>>>>> other "correlation" data in the response is redundant, I think.
>>>>>>
>>>>>> If the incoming request does not have OTel context info, what is
>> the
>>>>>> purpose of generating a Polaris-specific "correlation ID"? How is
>> it
>>>>>> envisioned to be used?
>>>>>>
>>>>>> If the intention is to correlate a Polaris response (operation)
>> with
>>>>> events
>>>>>> that might have resulted from its execution, I believe a more
>> robust
>>>>>> approach would be to propagate the OTel trace info (starting a new
>>>> trace
>>>>> if
>>>>>> necessary) into event data. Then, Polaris can also return the trace
>>>>> context
>>>>>> in the API response (top span). It's a bit awkward from the OTel
>>>>>> perspective, but might be an option for supporting custom
>>> correlators.
>>>> It
>>>>>> could be covered by a feature flag. The header name could be
>>>>>> "polaris-traceparent" for W3C Trace Context.
>>>>>>
>>>>>> Custom correlation code will be able to extract the trace ID from
>> the
>>>>>> response and from events and find related data. Granted, it will
>>>> require
>>>>> a
>>>>>> bit more effort for the custom code to decode trace IDs from the
>> OTel
>>>>>> context, but the format is standard and not complex. The benefit
>> for
>>>>>> Polaris, though, is that it can easily integrate with
>> OTel-compatible
>>>>>> observability platforms regardless of whether any particular
>>> deployment
>>>>>> uses custom correlators or not.
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>> Thanks,
>>>>>> Dmitri.
>>>>>>
>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <adutra@apache.org
>>>
>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Today, Polaris has the notion of "request ID", but its purpose is
>>> not
>>>>>>> entirely clear. It seems to serve as an observability feature to
>>>>>>> facilitate correlation. A pending PR aims to rename it to
>>>> "correlation
>>>>>>> ID" for better alignment with industry standards [1].
>>>>>>>
>>>>>>> However, this PR has brought to light overlaps with core
>> telemetry
>>>>>>> features: when OpenTelemetry (OTel) is enabled in Polaris, each
>>>>>>> request already has a trace ID and span ID, making a separate
>>>>>>> correlation ID redundant.
>>>>>>>
>>>>>>> Moreover, using the OTel trace ID and span ID in Polaris events,
>>>>>>> rather than the generated correlation ID, would significantly
>>>> simplify
>>>>>>> correlation of events with other traces.
>>>>>>>
>>>>>>> Therefore, I propose the following changes:
>>>>>>>
>>>>>>> 1. If OTel is enabled, use the trace ID and span ID as the
>>>> correlation
>>>>>>> ID for the request, instead of generating a random correlation
>> ID.
>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation ID header is
>>>> present
>>>>>>> in the request, use it.
>>>>>>> 3. If neither of the above conditions is met, generate a random
>>>>>>> correlation ID.
>>>>>>>
>>>>>>> I am somewhat undecided on the best approach when a correlation
>> ID
>>>>>>> header is present in the request. However, I believe it would be
>>> more
>>>>>>> sensible to disregard it if OTel is enabled, as OTel offers a
>> more
>>>>>>> robust solution for client-to-server trace propagation, e.g. W3C
>>>> Trace
>>>>>>> Context propagation [2].
>>>>>>>
>>>>>>> Please share your thoughts!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Alex
>>>>>>>
>>>>>>> [1]: https://www.google.com/url?q=https://github.com/apache/polaris/pull/2757&source=gmail-imap&ust=1761851831000000&usg=AOvVaw1-kAWfEk4tmsEg0q0GZBCn
>>>>>>> [2]: https://www.google.com/url?q=https://www.w3.org/TR/trace-context&source=gmail-imap&ust=1761851831000000&usg=AOvVaw22nMyOS7pbJ69XrBo5kHQS
>>>>>>>
>>>>>
>>>>
>>>
>>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Michael,
Logging x-request-id headers makes sense.
Just to confirm: if / when we restore that, Polaris will NOT generate new
IDs in case the header is not present in the request, correct?
I believe x-request-id can co-exist with OTel.
What about adding request IDs to events [1][2]? What's the intended use
case for that? Could you share some context here too?
Side note: I proposed [2877] flagging event persistence as "beta" in
1.2.0... This discussion adds another point towards that, I think.
[1]
https://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java#L97
[2]
https://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java#L66
[2877] https://github.com/apache/polaris/pull/2877#discussion_r2456300613
Thanks,
Dmitri.
On Thu, Oct 23, 2025 at 2:23 PM Michael Collado <co...@gmail.com>
wrote:
> Hey Dmitri
>
> The generating a request id is new code that was added after the original
> x-request-id support. You can see the state from ~1 year ago, we hard-coded
> request_id as the header we used for the MDC -
>
> https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java#L415-L416
> . At some point, it was changed to be configurable, then the
> ContextResolverFilter filter was refactored/eliminated and the
> RequestIdFilter took its responsibility, but lost some of its original
> functionality.
>
> My personal opinion is that restoring support for the x-request-id header
> is something that we should do, but if the header isn't present, falling
> back on simply using OTel trace ids is good enough (better, even) than
> generating another random request id.
>
> Mike
>
> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
> > Hi Michael,
> >
> > Thanks for the info!
> >
> > Working with Envoy's tracing headers makes sense to me. However, I
> wonder:
> > why would Polaris need to generate a new request ID inside its code?..
> > and return it in response headers?
> >
> > How important is it to propagate this ID to Polaris Events?
> >
> > I'm just trying to understand the full context :)
> >
> > Thanks,
> > Dmitri.
> >
> > On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <co...@gmail.com>
> > wrote:
> >
> > > I think the original intention for this requestId field was to support
> > > request propagation from load balancers, like Envoy (
> > >
> > >
> >
> https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing
> > > ), which is distinct from OTEL. Don't ask me why the default
> > > is Polaris-Request-Id - I think it was originally a custom thing, but
> > then
> > > we changed to integrate with existing conventions. Unfortunately,
> looking
> > > through the code, I think that the actual functional plumbing has been
> > lost
> > > in the course of multiple refactors around the call context and
> > resolver. I
> > > don't see references to that property or the underlying header.
> > >
> > > Support for the unofficial x-request-id header feels like something we
> > > should definitely keep, especially when Polaris is one service in a
> mesh
> > of
> > > services that maybe don't have OTel integration. I'm a fan of the OTel
> > > standard, but it's not entirely ubiquitous and there are many
> middleware
> > > layers that know how to forward on the x-request-id header.
> > >
> > > Mike
> > >
> > > On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
> > >
> > > > Yes, we should aim for interoperability with the existing de-facto
> > > > standard OTel and make it easy for users to integrate into their
> > > > observability platforms.
> > > >
> > > > On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <
> dimas@apache.org>
> > > > wrote:
> > > > >
> > > > > Hi Alex and All,
> > > > >
> > > > > I certainly support the idea of following OTel standards for
> > achieving
> > > > > "correlation" wrt Polaris requests and/or events.
> > > > >
> > > > > As to what form the correlation data should take, I believe it is
> > > > > conceptually what the OTel "context" represents. So, I believe it
> > makes
> > > > > sense for Polaris to support standard context propagators at the
> API
> > > > layer.
> > > > >
> > > > > If the incoming request has OTel context information, then
> returning
> > > any
> > > > > other "correlation" data in the response is redundant, I think.
> > > > >
> > > > > If the incoming request does not have OTel context info, what is
> the
> > > > > purpose of generating a Polaris-specific "correlation ID"? How is
> it
> > > > > envisioned to be used?
> > > > >
> > > > > If the intention is to correlate a Polaris response (operation)
> with
> > > > events
> > > > > that might have resulted from its execution, I believe a more
> robust
> > > > > approach would be to propagate the OTel trace info (starting a new
> > > trace
> > > > if
> > > > > necessary) into event data. Then, Polaris can also return the trace
> > > > context
> > > > > in the API response (top span). It's a bit awkward from the OTel
> > > > > perspective, but might be an option for supporting custom
> > correlators.
> > > It
> > > > > could be covered by a feature flag. The header name could be
> > > > > "polaris-traceparent" for W3C Trace Context.
> > > > >
> > > > > Custom correlation code will be able to extract the trace ID from
> the
> > > > > response and from events and find related data. Granted, it will
> > > require
> > > > a
> > > > > bit more effort for the custom code to decode trace IDs from the
> OTel
> > > > > context, but the format is standard and not complex. The benefit
> for
> > > > > Polaris, though, is that it can easily integrate with
> OTel-compatible
> > > > > observability platforms regardless of whether any particular
> > deployment
> > > > > uses custom correlators or not.
> > > > >
> > > > > WDYT?
> > > > >
> > > > > Thanks,
> > > > > Dmitri.
> > > > >
> > > > > On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <adutra@apache.org
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Today, Polaris has the notion of "request ID", but its purpose is
> > not
> > > > > > entirely clear. It seems to serve as an observability feature to
> > > > > > facilitate correlation. A pending PR aims to rename it to
> > > "correlation
> > > > > > ID" for better alignment with industry standards [1].
> > > > > >
> > > > > > However, this PR has brought to light overlaps with core
> telemetry
> > > > > > features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > > > > > request already has a trace ID and span ID, making a separate
> > > > > > correlation ID redundant.
> > > > > >
> > > > > > Moreover, using the OTel trace ID and span ID in Polaris events,
> > > > > > rather than the generated correlation ID, would significantly
> > > simplify
> > > > > > correlation of events with other traces.
> > > > > >
> > > > > > Therefore, I propose the following changes:
> > > > > >
> > > > > > 1. If OTel is enabled, use the trace ID and span ID as the
> > > correlation
> > > > > > ID for the request, instead of generating a random correlation
> ID.
> > > > > > 2. Otherwise, if a (Polaris-specific) correlation ID header is
> > > present
> > > > > > in the request, use it.
> > > > > > 3. If neither of the above conditions is met, generate a random
> > > > > > correlation ID.
> > > > > >
> > > > > > I am somewhat undecided on the best approach when a correlation
> ID
> > > > > > header is present in the request. However, I believe it would be
> > more
> > > > > > sensible to disregard it if OTel is enabled, as OTel offers a
> more
> > > > > > robust solution for client-to-server trace propagation, e.g. W3C
> > > Trace
> > > > > > Context propagation [2].
> > > > > >
> > > > > > Please share your thoughts!
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > [1]: https://github.com/apache/polaris/pull/2757
> > > > > > [2]: https://www.w3.org/TR/trace-context
> > > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Michael Collado <co...@gmail.com>.
Hey Dmitri
The generating a request id is new code that was added after the original
x-request-id support. You can see the state from ~1 year ago, we hard-coded
request_id as the header we used for the MDC -
https://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java#L415-L416
. At some point, it was changed to be configurable, then the
ContextResolverFilter filter was refactored/eliminated and the
RequestIdFilter took its responsibility, but lost some of its original
functionality.
My personal opinion is that restoring support for the x-request-id header
is something that we should do, but if the header isn't present, falling
back on simply using OTel trace ids is good enough (better, even) than
generating another random request id.
Mike
On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov <di...@apache.org>
wrote:
> Hi Michael,
>
> Thanks for the info!
>
> Working with Envoy's tracing headers makes sense to me. However, I wonder:
> why would Polaris need to generate a new request ID inside its code?..
> and return it in response headers?
>
> How important is it to propagate this ID to Polaris Events?
>
> I'm just trying to understand the full context :)
>
> Thanks,
> Dmitri.
>
> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <co...@gmail.com>
> wrote:
>
> > I think the original intention for this requestId field was to support
> > request propagation from load balancers, like Envoy (
> >
> >
> https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing
> > ), which is distinct from OTEL. Don't ask me why the default
> > is Polaris-Request-Id - I think it was originally a custom thing, but
> then
> > we changed to integrate with existing conventions. Unfortunately, looking
> > through the code, I think that the actual functional plumbing has been
> lost
> > in the course of multiple refactors around the call context and
> resolver. I
> > don't see references to that property or the underlying header.
> >
> > Support for the unofficial x-request-id header feels like something we
> > should definitely keep, especially when Polaris is one service in a mesh
> of
> > services that maybe don't have OTel integration. I'm a fan of the OTel
> > standard, but it's not entirely ubiquitous and there are many middleware
> > layers that know how to forward on the x-request-id header.
> >
> > Mike
> >
> > On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
> >
> > > Yes, we should aim for interoperability with the existing de-facto
> > > standard OTel and make it easy for users to integrate into their
> > > observability platforms.
> > >
> > > On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <di...@apache.org>
> > > wrote:
> > > >
> > > > Hi Alex and All,
> > > >
> > > > I certainly support the idea of following OTel standards for
> achieving
> > > > "correlation" wrt Polaris requests and/or events.
> > > >
> > > > As to what form the correlation data should take, I believe it is
> > > > conceptually what the OTel "context" represents. So, I believe it
> makes
> > > > sense for Polaris to support standard context propagators at the API
> > > layer.
> > > >
> > > > If the incoming request has OTel context information, then returning
> > any
> > > > other "correlation" data in the response is redundant, I think.
> > > >
> > > > If the incoming request does not have OTel context info, what is the
> > > > purpose of generating a Polaris-specific "correlation ID"? How is it
> > > > envisioned to be used?
> > > >
> > > > If the intention is to correlate a Polaris response (operation) with
> > > events
> > > > that might have resulted from its execution, I believe a more robust
> > > > approach would be to propagate the OTel trace info (starting a new
> > trace
> > > if
> > > > necessary) into event data. Then, Polaris can also return the trace
> > > context
> > > > in the API response (top span). It's a bit awkward from the OTel
> > > > perspective, but might be an option for supporting custom
> correlators.
> > It
> > > > could be covered by a feature flag. The header name could be
> > > > "polaris-traceparent" for W3C Trace Context.
> > > >
> > > > Custom correlation code will be able to extract the trace ID from the
> > > > response and from events and find related data. Granted, it will
> > require
> > > a
> > > > bit more effort for the custom code to decode trace IDs from the OTel
> > > > context, but the format is standard and not complex. The benefit for
> > > > Polaris, though, is that it can easily integrate with OTel-compatible
> > > > observability platforms regardless of whether any particular
> deployment
> > > > uses custom correlators or not.
> > > >
> > > > WDYT?
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> > > > On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <ad...@apache.org>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Today, Polaris has the notion of "request ID", but its purpose is
> not
> > > > > entirely clear. It seems to serve as an observability feature to
> > > > > facilitate correlation. A pending PR aims to rename it to
> > "correlation
> > > > > ID" for better alignment with industry standards [1].
> > > > >
> > > > > However, this PR has brought to light overlaps with core telemetry
> > > > > features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > > > > request already has a trace ID and span ID, making a separate
> > > > > correlation ID redundant.
> > > > >
> > > > > Moreover, using the OTel trace ID and span ID in Polaris events,
> > > > > rather than the generated correlation ID, would significantly
> > simplify
> > > > > correlation of events with other traces.
> > > > >
> > > > > Therefore, I propose the following changes:
> > > > >
> > > > > 1. If OTel is enabled, use the trace ID and span ID as the
> > correlation
> > > > > ID for the request, instead of generating a random correlation ID.
> > > > > 2. Otherwise, if a (Polaris-specific) correlation ID header is
> > present
> > > > > in the request, use it.
> > > > > 3. If neither of the above conditions is met, generate a random
> > > > > correlation ID.
> > > > >
> > > > > I am somewhat undecided on the best approach when a correlation ID
> > > > > header is present in the request. However, I believe it would be
> more
> > > > > sensible to disregard it if OTel is enabled, as OTel offers a more
> > > > > robust solution for client-to-server trace propagation, e.g. W3C
> > Trace
> > > > > Context propagation [2].
> > > > >
> > > > > Please share your thoughts!
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > [1]: https://github.com/apache/polaris/pull/2757
> > > > > [2]: https://www.w3.org/TR/trace-context
> > > > >
> > >
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Michael,
Thanks for the info!
Working with Envoy's tracing headers makes sense to me. However, I wonder:
why would Polaris need to generate a new request ID inside its code?..
and return it in response headers?
How important is it to propagate this ID to Polaris Events?
I'm just trying to understand the full context :)
Thanks,
Dmitri.
On Thu, Oct 23, 2025 at 1:29 PM Michael Collado <co...@gmail.com>
wrote:
> I think the original intention for this requestId field was to support
> request propagation from load balancers, like Envoy (
>
> https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing
> ), which is distinct from OTEL. Don't ask me why the default
> is Polaris-Request-Id - I think it was originally a custom thing, but then
> we changed to integrate with existing conventions. Unfortunately, looking
> through the code, I think that the actual functional plumbing has been lost
> in the course of multiple refactors around the call context and resolver. I
> don't see references to that property or the underlying header.
>
> Support for the unofficial x-request-id header feels like something we
> should definitely keep, especially when Polaris is one service in a mesh of
> services that maybe don't have OTel integration. I'm a fan of the OTel
> standard, but it's not entirely ubiquitous and there are many middleware
> layers that know how to forward on the x-request-id header.
>
> Mike
>
> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
>
> > Yes, we should aim for interoperability with the existing de-facto
> > standard OTel and make it easy for users to integrate into their
> > observability platforms.
> >
> > On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> > >
> > > Hi Alex and All,
> > >
> > > I certainly support the idea of following OTel standards for achieving
> > > "correlation" wrt Polaris requests and/or events.
> > >
> > > As to what form the correlation data should take, I believe it is
> > > conceptually what the OTel "context" represents. So, I believe it makes
> > > sense for Polaris to support standard context propagators at the API
> > layer.
> > >
> > > If the incoming request has OTel context information, then returning
> any
> > > other "correlation" data in the response is redundant, I think.
> > >
> > > If the incoming request does not have OTel context info, what is the
> > > purpose of generating a Polaris-specific "correlation ID"? How is it
> > > envisioned to be used?
> > >
> > > If the intention is to correlate a Polaris response (operation) with
> > events
> > > that might have resulted from its execution, I believe a more robust
> > > approach would be to propagate the OTel trace info (starting a new
> trace
> > if
> > > necessary) into event data. Then, Polaris can also return the trace
> > context
> > > in the API response (top span). It's a bit awkward from the OTel
> > > perspective, but might be an option for supporting custom correlators.
> It
> > > could be covered by a feature flag. The header name could be
> > > "polaris-traceparent" for W3C Trace Context.
> > >
> > > Custom correlation code will be able to extract the trace ID from the
> > > response and from events and find related data. Granted, it will
> require
> > a
> > > bit more effort for the custom code to decode trace IDs from the OTel
> > > context, but the format is standard and not complex. The benefit for
> > > Polaris, though, is that it can easily integrate with OTel-compatible
> > > observability platforms regardless of whether any particular deployment
> > > uses custom correlators or not.
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> > > On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Today, Polaris has the notion of "request ID", but its purpose is not
> > > > entirely clear. It seems to serve as an observability feature to
> > > > facilitate correlation. A pending PR aims to rename it to
> "correlation
> > > > ID" for better alignment with industry standards [1].
> > > >
> > > > However, this PR has brought to light overlaps with core telemetry
> > > > features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > > > request already has a trace ID and span ID, making a separate
> > > > correlation ID redundant.
> > > >
> > > > Moreover, using the OTel trace ID and span ID in Polaris events,
> > > > rather than the generated correlation ID, would significantly
> simplify
> > > > correlation of events with other traces.
> > > >
> > > > Therefore, I propose the following changes:
> > > >
> > > > 1. If OTel is enabled, use the trace ID and span ID as the
> correlation
> > > > ID for the request, instead of generating a random correlation ID.
> > > > 2. Otherwise, if a (Polaris-specific) correlation ID header is
> present
> > > > in the request, use it.
> > > > 3. If neither of the above conditions is met, generate a random
> > > > correlation ID.
> > > >
> > > > I am somewhat undecided on the best approach when a correlation ID
> > > > header is present in the request. However, I believe it would be more
> > > > sensible to disregard it if OTel is enabled, as OTel offers a more
> > > > robust solution for client-to-server trace propagation, e.g. W3C
> Trace
> > > > Context propagation [2].
> > > >
> > > > Please share your thoughts!
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > [1]: https://github.com/apache/polaris/pull/2757
> > > > [2]: https://www.w3.org/TR/trace-context
> > > >
> >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Michael Collado <co...@gmail.com>.
I think the original intention for this requestId field was to support
request propagation from load balancers, like Envoy (
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing
), which is distinct from OTEL. Don't ask me why the default
is Polaris-Request-Id - I think it was originally a custom thing, but then
we changed to integrate with existing conventions. Unfortunately, looking
through the code, I think that the actual functional plumbing has been lost
in the course of multiple refactors around the call context and resolver. I
don't see references to that property or the underlying header.
Support for the unofficial x-request-id header feels like something we
should definitely keep, especially when Polaris is one service in a mesh of
services that maybe don't have OTel integration. I'm a fan of the OTel
standard, but it's not entirely ubiquitous and there are many middleware
layers that know how to forward on the x-request-id header.
Mike
On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp <sn...@snazy.de> wrote:
> Yes, we should aim for interoperability with the existing de-facto
> standard OTel and make it easy for users to integrate into their
> observability platforms.
>
> On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> >
> > Hi Alex and All,
> >
> > I certainly support the idea of following OTel standards for achieving
> > "correlation" wrt Polaris requests and/or events.
> >
> > As to what form the correlation data should take, I believe it is
> > conceptually what the OTel "context" represents. So, I believe it makes
> > sense for Polaris to support standard context propagators at the API
> layer.
> >
> > If the incoming request has OTel context information, then returning any
> > other "correlation" data in the response is redundant, I think.
> >
> > If the incoming request does not have OTel context info, what is the
> > purpose of generating a Polaris-specific "correlation ID"? How is it
> > envisioned to be used?
> >
> > If the intention is to correlate a Polaris response (operation) with
> events
> > that might have resulted from its execution, I believe a more robust
> > approach would be to propagate the OTel trace info (starting a new trace
> if
> > necessary) into event data. Then, Polaris can also return the trace
> context
> > in the API response (top span). It's a bit awkward from the OTel
> > perspective, but might be an option for supporting custom correlators. It
> > could be covered by a feature flag. The header name could be
> > "polaris-traceparent" for W3C Trace Context.
> >
> > Custom correlation code will be able to extract the trace ID from the
> > response and from events and find related data. Granted, it will require
> a
> > bit more effort for the custom code to decode trace IDs from the OTel
> > context, but the format is standard and not complex. The benefit for
> > Polaris, though, is that it can easily integrate with OTel-compatible
> > observability platforms regardless of whether any particular deployment
> > uses custom correlators or not.
> >
> > WDYT?
> >
> > Thanks,
> > Dmitri.
> >
> > On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> > > Hi all,
> > >
> > > Today, Polaris has the notion of "request ID", but its purpose is not
> > > entirely clear. It seems to serve as an observability feature to
> > > facilitate correlation. A pending PR aims to rename it to "correlation
> > > ID" for better alignment with industry standards [1].
> > >
> > > However, this PR has brought to light overlaps with core telemetry
> > > features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > > request already has a trace ID and span ID, making a separate
> > > correlation ID redundant.
> > >
> > > Moreover, using the OTel trace ID and span ID in Polaris events,
> > > rather than the generated correlation ID, would significantly simplify
> > > correlation of events with other traces.
> > >
> > > Therefore, I propose the following changes:
> > >
> > > 1. If OTel is enabled, use the trace ID and span ID as the correlation
> > > ID for the request, instead of generating a random correlation ID.
> > > 2. Otherwise, if a (Polaris-specific) correlation ID header is present
> > > in the request, use it.
> > > 3. If neither of the above conditions is met, generate a random
> > > correlation ID.
> > >
> > > I am somewhat undecided on the best approach when a correlation ID
> > > header is present in the request. However, I believe it would be more
> > > sensible to disregard it if OTel is enabled, as OTel offers a more
> > > robust solution for client-to-server trace propagation, e.g. W3C Trace
> > > Context propagation [2].
> > >
> > > Please share your thoughts!
> > >
> > > Thanks,
> > > Alex
> > >
> > > [1]: https://github.com/apache/polaris/pull/2757
> > > [2]: https://www.w3.org/TR/trace-context
> > >
>
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Robert Stupp <sn...@snazy.de>.
Yes, we should aim for interoperability with the existing de-facto
standard OTel and make it easy for users to integrate into their
observability platforms.
On Wed, Oct 22, 2025 at 7:05 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Alex and All,
>
> I certainly support the idea of following OTel standards for achieving
> "correlation" wrt Polaris requests and/or events.
>
> As to what form the correlation data should take, I believe it is
> conceptually what the OTel "context" represents. So, I believe it makes
> sense for Polaris to support standard context propagators at the API layer.
>
> If the incoming request has OTel context information, then returning any
> other "correlation" data in the response is redundant, I think.
>
> If the incoming request does not have OTel context info, what is the
> purpose of generating a Polaris-specific "correlation ID"? How is it
> envisioned to be used?
>
> If the intention is to correlate a Polaris response (operation) with events
> that might have resulted from its execution, I believe a more robust
> approach would be to propagate the OTel trace info (starting a new trace if
> necessary) into event data. Then, Polaris can also return the trace context
> in the API response (top span). It's a bit awkward from the OTel
> perspective, but might be an option for supporting custom correlators. It
> could be covered by a feature flag. The header name could be
> "polaris-traceparent" for W3C Trace Context.
>
> Custom correlation code will be able to extract the trace ID from the
> response and from events and find related data. Granted, it will require a
> bit more effort for the custom code to decode trace IDs from the OTel
> context, but the format is standard and not complex. The benefit for
> Polaris, though, is that it can easily integrate with OTel-compatible
> observability platforms regardless of whether any particular deployment
> uses custom correlators or not.
>
> WDYT?
>
> Thanks,
> Dmitri.
>
> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hi all,
> >
> > Today, Polaris has the notion of "request ID", but its purpose is not
> > entirely clear. It seems to serve as an observability feature to
> > facilitate correlation. A pending PR aims to rename it to "correlation
> > ID" for better alignment with industry standards [1].
> >
> > However, this PR has brought to light overlaps with core telemetry
> > features: when OpenTelemetry (OTel) is enabled in Polaris, each
> > request already has a trace ID and span ID, making a separate
> > correlation ID redundant.
> >
> > Moreover, using the OTel trace ID and span ID in Polaris events,
> > rather than the generated correlation ID, would significantly simplify
> > correlation of events with other traces.
> >
> > Therefore, I propose the following changes:
> >
> > 1. If OTel is enabled, use the trace ID and span ID as the correlation
> > ID for the request, instead of generating a random correlation ID.
> > 2. Otherwise, if a (Polaris-specific) correlation ID header is present
> > in the request, use it.
> > 3. If neither of the above conditions is met, generate a random
> > correlation ID.
> >
> > I am somewhat undecided on the best approach when a correlation ID
> > header is present in the request. However, I believe it would be more
> > sensible to disregard it if OTel is enabled, as OTel offers a more
> > robust solution for client-to-server trace propagation, e.g. W3C Trace
> > Context propagation [2].
> >
> > Please share your thoughts!
> >
> > Thanks,
> > Alex
> >
> > [1]: https://github.com/apache/polaris/pull/2757
> > [2]: https://www.w3.org/TR/trace-context
> >
Re: [DISCUSS] Polaris correlation IDs and telemetry
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Alex and All,
I certainly support the idea of following OTel standards for achieving
"correlation" wrt Polaris requests and/or events.
As to what form the correlation data should take, I believe it is
conceptually what the OTel "context" represents. So, I believe it makes
sense for Polaris to support standard context propagators at the API layer.
If the incoming request has OTel context information, then returning any
other "correlation" data in the response is redundant, I think.
If the incoming request does not have OTel context info, what is the
purpose of generating a Polaris-specific "correlation ID"? How is it
envisioned to be used?
If the intention is to correlate a Polaris response (operation) with events
that might have resulted from its execution, I believe a more robust
approach would be to propagate the OTel trace info (starting a new trace if
necessary) into event data. Then, Polaris can also return the trace context
in the API response (top span). It's a bit awkward from the OTel
perspective, but might be an option for supporting custom correlators. It
could be covered by a feature flag. The header name could be
"polaris-traceparent" for W3C Trace Context.
Custom correlation code will be able to extract the trace ID from the
response and from events and find related data. Granted, it will require a
bit more effort for the custom code to decode trace IDs from the OTel
context, but the format is standard and not complex. The benefit for
Polaris, though, is that it can easily integrate with OTel-compatible
observability platforms regardless of whether any particular deployment
uses custom correlators or not.
WDYT?
Thanks,
Dmitri.
On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra <ad...@apache.org> wrote:
> Hi all,
>
> Today, Polaris has the notion of "request ID", but its purpose is not
> entirely clear. It seems to serve as an observability feature to
> facilitate correlation. A pending PR aims to rename it to "correlation
> ID" for better alignment with industry standards [1].
>
> However, this PR has brought to light overlaps with core telemetry
> features: when OpenTelemetry (OTel) is enabled in Polaris, each
> request already has a trace ID and span ID, making a separate
> correlation ID redundant.
>
> Moreover, using the OTel trace ID and span ID in Polaris events,
> rather than the generated correlation ID, would significantly simplify
> correlation of events with other traces.
>
> Therefore, I propose the following changes:
>
> 1. If OTel is enabled, use the trace ID and span ID as the correlation
> ID for the request, instead of generating a random correlation ID.
> 2. Otherwise, if a (Polaris-specific) correlation ID header is present
> in the request, use it.
> 3. If neither of the above conditions is met, generate a random
> correlation ID.
>
> I am somewhat undecided on the best approach when a correlation ID
> header is present in the request. However, I believe it would be more
> sensible to disregard it if OTel is enabled, as OTel offers a more
> robust solution for client-to-server trace propagation, e.g. W3C Trace
> Context propagation [2].
>
> Please share your thoughts!
>
> Thanks,
> Alex
>
> [1]: https://github.com/apache/polaris/pull/2757
> [2]: https://www.w3.org/TR/trace-context
>