Adding user principal tag in metrics

[Kostas Zoumpatianos <ko...@fivetran.com.INVALID>]

Hi team,

I recently opened a PR that optionally adds the user principal name as a
tag in metrics. This is useful for tracing API calls back to individual
users.

I understand that this can potentially expose information that people might
not want to necessarily expose, so this is why I set it to `false` by
default.

I am raising visibility on this with this email.

This is the associated issue: https://github.com/apache/polaris/issues/2444
and the associated PR: https://github.com/apache/polaris/pull/2445

Best regards,
Kostas

Re: Adding user principal tag in metrics

[Alexandre Dutra <ad...@apache.org>]

Hi Kostas,

Thank you for the PR and for opening this discussion thread!

I'm not against the proposed new metric tag, but I want to ensure
everyone understands the potential consequences.

Firstly, there's a risk of exposing sensitive information,
specifically the principal name, within metric tags. Currently, this
risk is minor because the principal name is internal to Polaris and
isn't considered Personally Identifiable Information (PII) – at least
that's my interpretation. However, this could change significantly in
the future once federated principals are fully integrated into
Polaris, as principal names could be things like personal emails.

Secondly, there is the risk of server OOM due to high cardinality
metric tags. We had a similar discussion back in May about the
"realm_id" metric tag [1]. While I note your proposal to add the
principal name only to API metrics (and not HTTP metrics), which is a
reasonable choice, I'm concerned that enabling both this new tag and
the existing "realm_id" tag in those metrics could amplify the
potential for DoS attacks. For instance, a deployment with 100 realms
and 1000 principals would likely be at risk.

Again, I'm not against the idea. However, I wonder if we shouldn't
implement a production readiness check to prevent users from
accidentally deploying a risky configuration. Wdyt?

Thanks,
Alex

[1]: https://lists.apache.org/thread/41y94pgzwgyff159tmo6kx5y8r6d0tgb


On Tue, Aug 26, 2025 at 5:33 AM Kostas Zoumpatianos
<ko...@fivetran.com.invalid> wrote:
>
> Hi team,
>
> I recently opened a PR that optionally adds the user principal name as a
> tag in metrics. This is useful for tracing API calls back to individual
> users.
>
> I understand that this can potentially expose information that people might
> not want to necessarily expose, so this is why I set it to `false` by
> default.
>
> I am raising visibility on this with this email.
>
> This is the associated issue: https://github.com/apache/polaris/issues/2444
> and the associated PR: https://github.com/apache/polaris/pull/2445
>
> Best regards,
> Kostas

Re: Adding user principal tag in metrics

[Kostas Zoumpatianos <ko...@fivetran.com.INVALID>]

Update: I added a production readiness check.

Kostas

On Tue, Aug 26, 2025 at 2:11 PM Kostas Zoumpatianos <
kostas.zoumpatianos@fivetran.com> wrote:

> Thank you Alexandre and Jean-Baptiste for your comments!
>
> So, if I could summarize the concerns they are the following:
> 1. Personally identifiable (sensitive) information. As Alexandre said this
> could become an issue once we get into federated principals territory.
> 2. OOMs, DOS attacks and metric data amplification especially if we
> compute the combinations between realms and principals.
>
> *Solutions*
> 1,2: For both I think a good practice is to create a configuration flag
> that is set to false by default and explicitly state the risks for the
> users when they enable it.
> I have such a flag in the PR and also set it to false by default.
> 2: What Alexandre proposed (i.e., a production readiness check) is
> something that I can easily add.
>
> Best,
> Kostas
>
> On Tue, Aug 26, 2025 at 11:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Kostas
>>
>> Thanks for starting the discussion.
>>
>> I think it makes sense to add user principal name in the metrics.
>> However, I have two comments:
>> 1. Some users could consider this as sensitive information.
>> Personally, as principal name is not a "composed" tag, I think it's
>> ok.
>> 2. It can generate a lot of "extra" data in the metrics.
>> For these two reasons, maybe we should consider a feature flag to
>> enable adding user principal name in the metrics, and not do it by
>> default.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On Tue, Aug 26, 2025 at 5:32 AM Kostas Zoumpatianos
>> <ko...@fivetran.com.invalid> wrote:
>> >
>> > Hi team,
>> >
>> > I recently opened a PR that optionally adds the user principal name as a
>> > tag in metrics. This is useful for tracing API calls back to individual
>> > users.
>> >
>> > I understand that this can potentially expose information that people
>> might
>> > not want to necessarily expose, so this is why I set it to `false` by
>> > default.
>> >
>> > I am raising visibility on this with this email.
>> >
>> > This is the associated issue:
>> https://github.com/apache/polaris/issues/2444
>> > and the associated PR: https://github.com/apache/polaris/pull/2445
>> >
>> > Best regards,
>> > Kostas
>>
>

Re: Adding user principal tag in metrics

[Kostas Zoumpatianos <ko...@fivetran.com.INVALID>]

Thank you Alexandre and Jean-Baptiste for your comments!

So, if I could summarize the concerns they are the following:
1. Personally identifiable (sensitive) information. As Alexandre said this
could become an issue once we get into federated principals territory.
2. OOMs, DOS attacks and metric data amplification especially if we compute
the combinations between realms and principals.

*Solutions*
1,2: For both I think a good practice is to create a configuration flag
that is set to false by default and explicitly state the risks for the
users when they enable it.
I have such a flag in the PR and also set it to false by default.
2: What Alexandre proposed (i.e., a production readiness check) is
something that I can easily add.

Best,
Kostas

On Tue, Aug 26, 2025 at 11:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:

> Hi Kostas
>
> Thanks for starting the discussion.
>
> I think it makes sense to add user principal name in the metrics.
> However, I have two comments:
> 1. Some users could consider this as sensitive information.
> Personally, as principal name is not a "composed" tag, I think it's
> ok.
> 2. It can generate a lot of "extra" data in the metrics.
> For these two reasons, maybe we should consider a feature flag to
> enable adding user principal name in the metrics, and not do it by
> default.
>
> Thoughts ?
>
> Regards
> JB
>
> On Tue, Aug 26, 2025 at 5:32 AM Kostas Zoumpatianos
> <ko...@fivetran.com.invalid> wrote:
> >
> > Hi team,
> >
> > I recently opened a PR that optionally adds the user principal name as a
> > tag in metrics. This is useful for tracing API calls back to individual
> > users.
> >
> > I understand that this can potentially expose information that people
> might
> > not want to necessarily expose, so this is why I set it to `false` by
> > default.
> >
> > I am raising visibility on this with this email.
> >
> > This is the associated issue:
> https://github.com/apache/polaris/issues/2444
> > and the associated PR: https://github.com/apache/polaris/pull/2445
> >
> > Best regards,
> > Kostas
>

Re: Adding user principal tag in metrics

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Hi Kostas

Thanks for starting the discussion.

I think it makes sense to add user principal name in the metrics.
However, I have two comments:
1. Some users could consider this as sensitive information.
Personally, as principal name is not a "composed" tag, I think it's
ok.
2. It can generate a lot of "extra" data in the metrics.
For these two reasons, maybe we should consider a feature flag to
enable adding user principal name in the metrics, and not do it by
default.

Thoughts ?

Regards
JB

On Tue, Aug 26, 2025 at 5:32 AM Kostas Zoumpatianos
<ko...@fivetran.com.invalid> wrote:
>
> Hi team,
>
> I recently opened a PR that optionally adds the user principal name as a
> tag in metrics. This is useful for tracing API calls back to individual
> users.
>
> I understand that this can potentially expose information that people might
> not want to necessarily expose, so this is why I set it to `false` by
> default.
>
> I am raising visibility on this with this email.
>
> This is the associated issue: https://github.com/apache/polaris/issues/2444
> and the associated PR: https://github.com/apache/polaris/pull/2445
>
> Best regards,
> Kostas