[DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

["Arsenault, Reginald P. via dev" <de...@polaris.apache.org>]

UNCLASSIFIED / NON CLASSIFI?

Hi! As per our findings in [2], we are looking to enable connectivity with our on-prem S3 storage appliance, which requires changes to how Polaris connects to S3. Currently, the way Polaris connects to S3 does not work for our on-prem S3 instance because of the way it expects to connect to the STS server (i.e. without any specific authentication).
We propose enhancing the AwsCredentialsStorageIntegration to support a more flexible authorization mechanism:

  *   Goal: We want the ability to pass the requesting user's identity (likely a token) to the STS. This would allow the STS to make fine-grained decisions on what the user can and cannot do based on their specific identity.
  *   Proposed Implementation: We believe this functionality can be achieved by adding an option within AwsCredentialsStorageIntegration [1] to connect using the AssumeRoleWithWebIdentity flow.
We would greatly appreciate some guidance on the following changes before we begin development:

  1.  What is the recommended or existing mechanism within Polaris for capturing and passing along an authenticated user's identity token to the AwsCredentialsStorageIntegration component, so we can pass it to the STS? I could be wrong, but I believe the token that gets passed to STS doesn't need to be scoped for the STS, so it could just be whatever token is given to Polaris by the user.
  2.  How should we best modify the AwsCredentialsStorageIntegration class and configuration to be able to handle both the existing AssumeRole use case and the new AssumeRoleWithWebIdentity use case?
Any pointers or suggestions on the best place to inject the user token and manage the configuration switch would be helpful.

[1] https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L107
[2] https://github.com/apache/polaris/issues/3038

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Dmitri Bourlatchkov <di...@apache.org>]

Hi Reginald,

The JWT token would normally be handled by Quarkus (or its plugins).... but
it should be possible to access it in application (Polaris) code too.

Have a look at OidcTenantResolvingAugmentor [1] and related classes.

We have PolarisPrincipal to represent API subjects. It might be reasonable
to copy the JWT into it, following the pattern
of javax.security.auth.Subject.getPrivateCredentials()... Might as well
have something specific to JWT because these days other credential types
are not likely to be used in practice... at least not in Polaris.

[1]
https://github.com/apache/polaris/blob/96f1459de5dd41dcfd5496e6a4f01810c4868526/runtime/service/src/main/java/org/apache/polaris/service/auth/external/tenant/OidcTenantResolvingAugmentor.java#L62

Cheers,
Dmitri.

On Fri, Nov 21, 2025 at 9:58 AM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFIÉ
>
> Thanks for the initial guidance Dmitri! I've gotten as far as configuring
> the class to handle either use-case, but I can't for the life of me find
> anything that will allow me to get a users token! Getting the
> securityContext into the AwsCredentialsStorageIntegration is rather easy,
> but the securityContext can't provide a token for the user. Does anyone
> have any suggestions for how to actually get the requesting users token?
> Here's my rough draft so far -
> https://github.com/cccs-cat001/polaris/commit/7d212033145a0a3f1dd3fa7137b35ace114bfcfc
>
> Thanks,
> R.J.
>

RE: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

["Arsenault, Reginald P. via dev" <de...@polaris.apache.org>]

UNCLASSIFIED / NON CLASSIFIÉ

Hi! I opened the PR [1] to address this feature. Please take a look at it when you have the time 😊

[1] https://github.com/apache/polaris/pull/3170

Thanks,
R.J.

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Dmitri Bourlatchkov <di...@apache.org>]

Hi R.J.,

Another point to consider: Polaris may need access to storage from async /
background tasks [1].

This probably needs some thought too in the context of using API user's
credentials for storage access. The critical point here is that although
all tasks run in the same JVM now, they may become distributed later [2].

It is relevant to "DROP TABLE PURGE" requests, but may be needed in other
contexts too (I'm not completely up-to-date on this code).

[1]
https://github.com/apache/polaris/blob/d03c7174ccf6771a0673e02b5e6d691d6cca9419/runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java#L58

[2] https://lists.apache.org/thread/gg0kn89vmblmjgllxn7jkn8ky2k28f5l

Cheers,
Dmitri.

On Fri, Nov 21, 2025 at 9:58 AM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFIÉ
>
> Thanks for the initial guidance Dmitri! I've gotten as far as configuring
> the class to handle either use-case, but I can't for the life of me find
> anything that will allow me to get a users token! Getting the
> securityContext into the AwsCredentialsStorageIntegration is rather easy,
> but the securityContext can't provide a token for the user. Does anyone
> have any suggestions for how to actually get the requesting users token?
> Here's my rough draft so far -
> https://github.com/cccs-cat001/polaris/commit/7d212033145a0a3f1dd3fa7137b35ace114bfcfc
>
> Thanks,
> R.J.
>

RE: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

["Arsenault, Reginald P. via dev" <de...@polaris.apache.org>]

UNCLASSIFIED / NON CLASSIFIÉ

Thanks for the initial guidance Dmitri! I've gotten as far as configuring the class to handle either use-case, but I can't for the life of me find anything that will allow me to get a users token! Getting the securityContext into the AwsCredentialsStorageIntegration is rather easy, but the securityContext can't provide a token for the user. Does anyone have any suggestions for how to actually get the requesting users token? Here's my rough draft so far - https://github.com/cccs-cat001/polaris/commit/7d212033145a0a3f1dd3fa7137b35ace114bfcfc

Thanks,
R.J.

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Dmitri Bourlatchkov <di...@apache.org>]

Hi Reginald,

Just a rough sketch:

* AwsCredentialsStorageIntegration is currently able to take an
AwsCredentialsProvider

* AwsCredentialsStorageIntegration can use AwsStorageConfigurationInfo to
decide how exactly to obtain storage credentials. It should be possible to
add settings there to switch to AssumeRoleWithWebIdentity

* However, currently AwsCredentialsProvider is hard-wired to static
credentials [1]

* StorageAccessConfigProvider is request-scoped, so in principle it should
be able to feed the authenticated principal's credentials into the storage
integration code. However, this needs a bit of code archeology... There
will be some rough edges there :) If you're willing to explore this path in
the code, I think that's where a connection to the API user could be made.

[1]
https://github.com/apache/polaris/blob/20febdaede19fb7c46e120652fdd1a262c2138e4/runtime/service/src/main/java/org/apache/polaris/service/storage/StorageConfiguration.java#L89

Cheers,
Dmitri.

On Thu, Nov 20, 2025 at 2:32 PM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFI?
>
> Hi! As per our findings in [2], we are looking to enable connectivity with
> our on-prem S3 storage appliance, which requires changes to how Polaris
> connects to S3. Currently, the way Polaris connects to S3 does not work for
> our on-prem S3 instance because of the way it expects to connect to the STS
> server (i.e. without any specific authentication).
> We propose enhancing the AwsCredentialsStorageIntegration to support a
> more flexible authorization mechanism:
>
>   *   Goal: We want the ability to pass the requesting user's identity
> (likely a token) to the STS. This would allow the STS to make fine-grained
> decisions on what the user can and cannot do based on their specific
> identity.
>   *   Proposed Implementation: We believe this functionality can be
> achieved by adding an option within AwsCredentialsStorageIntegration [1] to
> connect using the AssumeRoleWithWebIdentity flow.
> We would greatly appreciate some guidance on the following changes before
> we begin development:
>
>   1.  What is the recommended or existing mechanism within Polaris for
> capturing and passing along an authenticated user's identity token to the
> AwsCredentialsStorageIntegration component, so we can pass it to the STS? I
> could be wrong, but I believe the token that gets passed to STS doesn't
> need to be scoped for the STS, so it could just be whatever token is given
> to Polaris by the user.
>   2.  How should we best modify the AwsCredentialsStorageIntegration class
> and configuration to be able to handle both the existing AssumeRole use
> case and the new AssumeRoleWithWebIdentity use case?
> Any pointers or suggestions on the best place to inject the user token and
> manage the configuration switch would be helpful.
>
> [1]
> https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L107
> [2] https://github.com/apache/polaris/issues/3038
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Adnan Hemani via dev <de...@polaris.apache.org>]

Hi Robert and all,

I think your analogy here encompasses the entire idea as to why this
feature should not be supported as it is written today in Polaris:

> On the other hand, it doesn't make much sense to lock the door if you can
walk into the house through the open garage.

To extend this analogy, the correct question asked next should be: why is
the garage open - and shouldn't we be recommending people to close
their garages? And not: Let's open the windows too while we're at it. I'm
not looking for a "perfect security model" - but one that mirrors the
security model we have for the "AssumeRole" path today.

Per my reading of your email (correct me if I'm missing something, since it
was a long mail with lots of loosely-related points), you are making the
assumption that users who are interested in using AssumeRoleWithWebIdentity
are broadly interested in managing storage credentials directly on a
per-principal/per-user basis. As I said on this thread before, this would
be a sharp departure from the model that is used in Polaris' AssumeRole
workflow. If this assumption is correct, then I agree, passing user tokens
directly to the "STS" is the way forward - but then Polaris should either
1) not attempt to assert its own authorization on a table-level at all for
such use cases OR 2) ensure that STS permissions are taken into account
before returning back to the user. In either case, the administrators of
such a system have made their decision that they would like to manage
storage access elsewhere and Polaris should be respecting that.

But I personally don't think this is the direction that *all* users of
AssumeRoleWithWebIdentity would like. A big factor in why users choose
Polaris is the governance features it provides - and the simplicity it
provides for managing permissions in one place. We should be working
towards something like Point 2 from your message to ensure that
AssumeRoleWithWebIdentity works *alongside* Polaris authorization and have
that as the "native" support that Polaris provides for
AssumeRoleWithWebIdentity.

Now, to keep the conversation moving forward, I can think of the following
options instead:
1. [Not recommended] We don't support this use case. We recommend users who
have such a setup to disable credential vending and then make a change to
the Iceberg client to go to "STS" directly for storage access.
2. [Recommended] We implement a "hybrid-mode" of External PDP where we
conditionally delegate authorization to "STS" if we need.
    a) In further detail: we would create a new PolarisAuthorizer (similar
to the OpaPolarisAuthorizer
<https://github.com/apache/polaris/blob/main/extensions/auth/opa/impl/src/main/java/org/apache/polaris/extension/auth/opa/OpaPolarisAuthorizer.java>)
that will delegate all non-credential vending / non-table metadata
authorization calls to the OOTB PolarisAuthorizer and if credential vending
and/or table metadata is required, will send the request to STS instead.
3. [Not recommended for the short-term] Implement OAuth delegation support
to Polaris. Although it is a good option in the long-term, as others have
said in this thread, it may be a lot more involved than Option 2 will be
and so may not be the first choice to attract new users quickly. But I'm in
agreement that this is the right direction in the long-term to support the
AssumeRoleWithWebIdentity use case using a proper "trust" model.

Best,
Adnan Hemani


On Tue, Dec 9, 2025 at 9:39 AM Tornike Gurgenidze <to...@gmail.com>
wrote:

> I also think the "bypassing polaris" problem is unavoidable, you
> effectively already have a similar situation with credential vending
> disabled. The thing that bothered me more initially is that this flow feels
> exactly like using polaris with no credentials vending, the only difference
> being that sts call is executed on the server rather than on the client.
>
> Still, having thought about it a bit more, I think there's another way to
> look at it that makes more sense (at least to me :)). The other important
> difference from vendingless model is that server-side async tasks (like
> create table, purge table) will now be executed with client credentials
> rather than server credentials, effectively meaning that server doesn't
> really need to have any privileges on the storage. You basically end up
> having 3 different security models:
>
> 1. Vending disabled: both server and client need storage permissions
> 2. Vending enabled (AssumeRole): Only the server needs permissions
> 3. Vending enabled (AssumeRoleWithWebIdentity): Only the client needs
> permissions
>
> I think another important point is that when using
> AssumeRoleWithWebIdentity, RoleArn configured server-side should always be
> ignored (or ideally not configured in the first place). This was what
> confused me the most in the PR, it looked like a user token was being used
> to assume the configured storage role (which normally has very broad
> privileges). Instead of passing the role, I'm assuming the token itself
> will have some way of carrying its own storage privileges (something like
> this token claim example
> <
> https://docs.min.io/enterprise/aistor-object-store/administration/iam/access/oidc-access/#json-web-token-claim
> >
> in minio).
>
> On Tue, Dec 9, 2025 at 3:42 PM Robert Stupp <sn...@snazy.de> wrote:
>
> > Hi all,
> >
> > Seeing such a big engagement is amazing! Having so many people care
> > about security and user oriented features is absolutely awesome for
> > the Polaris project!
> >
> > Truth to be told: security is relative and needs to be considered in a
> > holistic way. Security can never be 100% perfect (it's also software).
> > It is totally valid and actually required to uphold strong and strict
> > security standards, wherever possible and wherever it makes sense.
> > This is what I fully support!
> > On the other hand, it doesn't make much sense to lock the door if you
> > can walk into the house through the open garage.
> >
> > Full disclosure: I'll deviate into very related aspects and pieces to
> > cover the "whole picture".
> >
> > From the above messages, it feels like there is some confusion about
> > what Polaris can fully control/enforce and what Polaris can only
> > "recommend" due to the API and architecture and libraries we and our
> > users have to use. More on this below.
> >
> > The term "privilege escalation" in the previous messages caught my
> > attention.
> > A "privilege escalation" is a situation when a user can gain *more* or
> > *higher* access privileges (against the same or a different system).
> > In this AssumeRoleWithWebIdentity case, the user already has the
> > privileges. Nothing that's been described gives a user *more* or
> > *higher* access privileges.
> > Polaris returns (potentially down-scoped) credentials leveraging IAM
> > policies, but those can already be overridden by users with different
> > credentials. This allows a (bad?) client, for example, to write data
> > to arbitrary locations, effectively bypassing the restrictions of
> > down-scoped credentials to enforce table locations. But even that can
> > already be worked around using standard configuration options (think:
> > using write.folder-storage.path or write.object-storage.path).
> > For a "smoother experience", it's also possible for users to extend
> > the S3FileIO to automatically retrieve and refresh the object storage
> > credentials directly via STS/AssumeRoleWithWebIdentity - just override
> > the file-io configuration to use it. It probably wouldn't even count
> > as a security issue, because it does not work around the already
> > granted access privileges.
> > All these _standard_ configuration options and mechanisms are the root
> > cause, but it is neither a new issue nor a Polaris issue nor can
> > Polaris do anything to prevent this from ever happening.
> >
> > A different previously mentioned aspect that I'd like to highlight:
> > There is currently no way to gain insights into *who* did *what*
> > against the object store, because the object storage credentials that
> > are currently vended/issued by Polaris are _not_ linked to a
> > user/client identity. AssumeRoleWithWebIdentity seems to enable
> > getting this very valuable information, which can be necessary for
> > certain legal/governance requirements.
> >
> > Looking a bit around in the ecosystem (disclaimer: I haven't looked at
> > the nitty gritty details), Trino and Hive already support S3 via
> > AssumeRoleWithWebIdentity.
> >
> > As already mentioned in the previous messages, "trust" is the key for
> > the STS/AssumeRoleWithWebIdentity scenario.
> > The STS service should ideally require a token issued for Polaris on
> > behalf of the user ("delegation"). That's a legit scenario and a very
> > valuable feature for Polaris, in addition to the request to support
> > AssumeRoleWithWebIdentity. (Mentioned in the previous messages.)
> >
> > The "normal" workflow would be: client -> Polaris -> Polaris gets
> > down-scoped credentials from STS -> Polaris returns (potentially
> > down-scoped) credentials to the client.
> > Again, a client can also call STS directly to get S3 credentials via
> > AssumeRoleWithWebIdentity, and work with *any* tool directly against
> > S3.
> > There is nothing that can prevent this from happening, as long as the
> > STS service is accessible and STS accepts the JWT ("trust").
> >
> > Technically and assuming that the user has the necessary access
> > privileges, all operations, even those that effectively bypass
> > validations and checks in Polaris, are possible.
> > This is, again, not a problem with AssumeRoleWithWebIdentity, but a
> > general and not a Polaris specific problem. That the source of truth
> > (table/view metadata) is in the object storage contributes to the
> > situation. Plus: that the client generates the path names and the
> > contents of all meta information.
> >
> > Polaris does govern access to the meta information about namespaces
> > and tables and views. For the latter it is, beside some additional
> > properties, just the pointer to the table/view metadata. If a user has
> > the necessary privileges, overwriting table/view metadata is possible.
> >
> > What I read between the lines in some of the previous messages is an
> > ask for a "perfect security model". Due to the API and the
> > architecture we have to work with, it is IMO not possible. It might
> > help to generally restrict access to the object store to totally
> > controlled systems in an isolated security enclave, which is a very
> > different thing.
> >
> > My proposal is to enable _all_ Polaris users to gain the value add of
> > the original proposal, and not push our users towards maintaining hard
> > forks for general-purpose functionality:
> >
> > 1. Continue the AssumeRoleWithWebIdentity proposal, opt-in via a
> > catalog or realm level configuration, maybe guarded with a "production
> > ready warning", if people really insist
> > 2. Add OAuth delegation support to Polaris, starting with the
> > AssumeRoleWithWebIdentity use case
> >    2.a) Polaris tokens: re-issue the token with a specific claim trusted
> > by STS
> >    2.b) Delegation for "proper" IdPs
> >
> > Yes, 1 without 2 might be a concern.
> >
> > How does that sound?
> >
> > Robert
> >
> > On Sat, Dec 6, 2025 at 1:46 AM Adnan Hemani via dev
> > <de...@polaris.apache.org> wrote:
> > >
> > > Hi all,
> > >
> > > I can get behind Yufei's idea - I don't see any security concerns with
> > it.
> > > In this case, the admin is explicitly electing (at an application
> level)
> > to
> > > outsource their AuthZ to an external system and there still remains a
> > > single source of truth of which principals are allowed to access what.
> To
> > > put it in explicit terms, Polaris authorization should be *fully
> > disabled*
> > > for any catalog that elects this model - and we *must* enforce
> disabling
> > > Polaris' own authorization checks to prevent users being misled and/or
> > > confused by this feature.
> > >
> > > But I also agree with Laurent's analysis of where the OIDC/JWT world is
> > > going today - and I think we should invest in supporting that workflow
> > in a
> > > similar manner to what Laurent is expressing. That model will allow
> users
> > > to still use Polaris authorization *AND* AssumeRoleWithWebIdentity APIs
> > for
> > > their on-prem (or even cloud) platforms in a secure, non-confusing
> > manner -
> > > which is the ideal scenario to me.
> > >
> > > However for RJ's use case, the question remains, will this on-prem
> "STS"
> > > system be able to integrate with the External Policy Decision Point
> (PDP
> > > <
> >
> https://polaris.apache.org/in-dev/unreleased/managing-security/external-pdp/
> > >)
> > > model?
> > > I have no background in what system is being used, so it's hard for me
> to
> > > tell.
> > >
> > > Best,
> > > Adnan Hemani
> > >
> > > On Fri, Dec 5, 2025 at 10:09 AM Yufei Gu <fl...@gmail.com> wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > To add one more perspective here, this is not only a security concern
> > but
> > > > also a major usability and mental-model concern for Polaris users.
> > Here is
> > > > an example:
> > > >
> > > >    - Alice has read + write privileges at the STS/storage layer.
> > > >    - In Polaris, Alice is intentionally granted read-only access to a
> > > > table.
> > > >
> > > > Under the proposed model, because the user JWT is forwarded directly
> to
> > > > STS, Alice can simply bypass Polaris and call STS directly. STS then
> > > > (correctly, from its own viewpoint) grants her write permission,
> since
> > it
> > > > has no awareness of Polaris’ table-level authorization.
> > > >
> > > > This results in *privilege escalation: *Polaris says “read-only,” but
> > Alice
> > > > can still perform writes by going around Polaris.
> > > >
> > > > Beyond the security implications, this is extremely confusing for
> > admins
> > > > and users. Polaris presents one authoritative access decision, but
> the
> > > > underlying storage enforces another. Users cannot reasonably be
> > expected to
> > > > understand why they can “write when Polaris says they can’t,”, and
> > > > why audit logs and enforcement don’t align.
> > > >
> > > > *A possible path forward:* one direction we can take, which keeps
> your
> > use
> > > > case viable while also preserving Polaris’ governance guarantees, is
> to
> > > > introduce an OPA-style authorization module:
> > > >
> > > >    - Polaris can delegate authZ decisions to a remote system (e.g.,
> > OPA,
> > > >    your existing STS-side policy engine, etc).
> > > >    - Credential vending still occurs in Polaris, but only after the
> > > >    delegated AuthZ result has been evaluated.
> > > >
> > > > This approach allows deployments like yours to centralize their
> > existing
> > > > rules in the external system, while also ensuring:
> > > >
> > > >    - No privilege escalation occurs by bypassing Polaris
> > > >    - Credential vending always honors Polaris’ (or delegated) AuthZ
> > > > decision
> > > >    - We avoid creating two conflicting interpretations of the same
> > user’s
> > > >    permissions
> > > >
> > > > This feels like a direction that could satisfy both RJ’s on-prem
> > scenario
> > > > and the broader community’s expectations around governance semantics.
> > > >
> > > > Happy to continue iterating on how such a module would be wired in,
> but
> > > > conceptually it gives us a clean separation: AuthZ may be delegated,
> > but
> > > > enforcement remains consistent and predictable through Polaris.
> > > >
> > > > Yufei
> > > >
> > > >
> > > > On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <dimas@apache.org
> >
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I guess my comment in GH regarding Polaris-owned tokens was not
> > precise
> > > > > enough. Apologies for the confusion and I'll try to clarify below.
> > > > >
> > > > > As far as I understand, in RJ's use case, Polaris is actually
> > expected to
> > > > > pass the user's auth token from Polaris API to STS. Arguably, it
> is a
> > > > very
> > > > > specific use case and may be applicable only to some very specific
> > > > on-prem
> > > > > environments (please correct me if I misinterpret that), but I
> > believe it
> > > > > is fine to support.
> > > > >
> > > > > In that use case Polaris acts as a Resource server from the OAuth2
> > > > > perspective and no extra auth tokens are necessary.
> > > > >
> > > > > Outside of RJ's use case (and PR), I do not really object to
> > supporting
> > > > > OAuth2 token exchange in Polaris. It could be a valuable feature,
> but
> > > > from
> > > > > my POV it is a totally separate feature (not related to [3170])
> > > > >
> > > > > [3170] https://github.com/apache/polaris/pull/3170
> > > > >
> > > > > Cheers,
> > > > > Dmitri.
> > > > >
> > > > > On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> > > > > dev@polaris.apache.org> wrote:
> > > > >
> > > > > > UNCLASSIFIED / NON CLASSIFIÉ
> > > > > >
> > > > > > Hi Laurent, Thanks for your questions!
> > > > > >
> > > > > > > I'm a big fan of those token exchange scenarios which avoid to
> > create
> > > > > > long lived secrets to interact between systems, and rely instead
> of
> > > > > setting
> > > > > > trust mechanisms between them.
> > > > > >
> > > > > > We are too, which is why can’t go with the standard
> > > > > > access-key-id/access-key-secret setup that a regular AWS S3
> system
> > > > uses.
> > > > > By
> > > > > > passing along the short lived JWT token of the user to the STS,
> we
> > can
> > > > > get
> > > > > > a new, short-lived, access-key-id/access-key-secret for our
> > on-prem S3
> > > > > > buckets with every request. Having one generic service account
> > doing
> > > > all
> > > > > > the operations on the S3 for all our users is a security risk for
> > us.
> > > > > >
> > > > > >
> > > > > > > My main question is that if the authorization token sent to
> > polaris
> > > > is
> > > > > > passed as-is to the STS server, what happens if the client
> directly
> > > > sends
> > > > > > the token to the STS system directly?
> > > > > >
> > > > > > For our on-prem S3/STS setup we have no issue with users
> accessing
> > STS
> > > > > > System directly. It is normal for users to be able to access the
> > > > buckets.
> > > > > > The STS checks the permissions users have on the buckets they’re
> > > > > > requesting, so there is no way for a user to gain more privileges
> > to
> > > > > > potentially read/write a bucket  they don’t get access to through
> > > > > polaris,
> > > > > > by going directly to the STS. If anything if a user going
> directly
> > to
> > > > the
> > > > > > STS they’d get MORE permissions, which is totally fine and
> > completely
> > > > > > reasonable.
> > > > > >
> > > > > >
> > > > > > > You probably have thought of that scenario already so you may
> > have
> > > > some
> > > > > > contingency system in place or maybe it's not a actual issue in
> > your
> > > > > > deployment situation?
> > > > > >
> > > > > > We heavily audit what the users are doing. Polaris will log “this
> > user
> > > > is
> > > > > > requesting credentials for this table”, the STS logs “this user
> is
> > > > > > requesting credentials for this bucket” and our on-prem S3 logs
> > “this
> > > > > user
> > > > > > did <thing> to this bucket.” There is no concern from us over
> > someone
> > > > > > potentially skirting privileges.
> > > > > >
> > > > > >
> > > > > > > One idea I had to prevent user to bypass Polaris authorization
> > system
> > > > > > would be for Polaris itself to issue its own JWT tokens, and when
> > > > > accessing
> > > > > > storage, Polaris would do an internal exchange where claims from
> > the
> > > > > > inbound token (received by Polaris) would be copied over to the
> > > > outbound
> > > > > > token (sent to STS), after proper validation of course. Is it a
> > > > scenario
> > > > > > you have considered by any chance?
> > > > > >
> > > > > > From what I gather, Polaris has no intentions on being in the
> > business
> > > > of
> > > > > > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have
> > no
> > > > > issue
> > > > > > with users going around Polaris to access the STS and S3, this is
> > by
> > > > > > design. There is no guarantee that this feature will work for
> > AWS/Minio
> > > > > > [2], it will enable Polaris to support on-prem S3 use cases like
> > [3],
> > > > > > making it more widely available for everyone.
> > > > > >
> > > > > > [1]
> > > > https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > > > > > [2]
> > > > https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > > > > > [3] https://github.com/apache/polaris/issues/3038
> > > > > >
> > > > > > Thank you for your feedback and please let me know if there’s any
> > more
> > > > > > questions!
> > > > > >
> > > > > > Thanks,
> > > > > > R.J.
> > > > > >
> > > > >
> > > >
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Tornike Gurgenidze <to...@gmail.com>]

I also think the "bypassing polaris" problem is unavoidable, you
effectively already have a similar situation with credential vending
disabled. The thing that bothered me more initially is that this flow feels
exactly like using polaris with no credentials vending, the only difference
being that sts call is executed on the server rather than on the client.

Still, having thought about it a bit more, I think there's another way to
look at it that makes more sense (at least to me :)). The other important
difference from vendingless model is that server-side async tasks (like
create table, purge table) will now be executed with client credentials
rather than server credentials, effectively meaning that server doesn't
really need to have any privileges on the storage. You basically end up
having 3 different security models:

1. Vending disabled: both server and client need storage permissions
2. Vending enabled (AssumeRole): Only the server needs permissions
3. Vending enabled (AssumeRoleWithWebIdentity): Only the client needs
permissions

I think another important point is that when using
AssumeRoleWithWebIdentity, RoleArn configured server-side should always be
ignored (or ideally not configured in the first place). This was what
confused me the most in the PR, it looked like a user token was being used
to assume the configured storage role (which normally has very broad
privileges). Instead of passing the role, I'm assuming the token itself
will have some way of carrying its own storage privileges (something like
this token claim example
<https://docs.min.io/enterprise/aistor-object-store/administration/iam/access/oidc-access/#json-web-token-claim>
in minio).

On Tue, Dec 9, 2025 at 3:42 PM Robert Stupp <sn...@snazy.de> wrote:

> Hi all,
>
> Seeing such a big engagement is amazing! Having so many people care
> about security and user oriented features is absolutely awesome for
> the Polaris project!
>
> Truth to be told: security is relative and needs to be considered in a
> holistic way. Security can never be 100% perfect (it's also software).
> It is totally valid and actually required to uphold strong and strict
> security standards, wherever possible and wherever it makes sense.
> This is what I fully support!
> On the other hand, it doesn't make much sense to lock the door if you
> can walk into the house through the open garage.
>
> Full disclosure: I'll deviate into very related aspects and pieces to
> cover the "whole picture".
>
> From the above messages, it feels like there is some confusion about
> what Polaris can fully control/enforce and what Polaris can only
> "recommend" due to the API and architecture and libraries we and our
> users have to use. More on this below.
>
> The term "privilege escalation" in the previous messages caught my
> attention.
> A "privilege escalation" is a situation when a user can gain *more* or
> *higher* access privileges (against the same or a different system).
> In this AssumeRoleWithWebIdentity case, the user already has the
> privileges. Nothing that's been described gives a user *more* or
> *higher* access privileges.
> Polaris returns (potentially down-scoped) credentials leveraging IAM
> policies, but those can already be overridden by users with different
> credentials. This allows a (bad?) client, for example, to write data
> to arbitrary locations, effectively bypassing the restrictions of
> down-scoped credentials to enforce table locations. But even that can
> already be worked around using standard configuration options (think:
> using write.folder-storage.path or write.object-storage.path).
> For a "smoother experience", it's also possible for users to extend
> the S3FileIO to automatically retrieve and refresh the object storage
> credentials directly via STS/AssumeRoleWithWebIdentity - just override
> the file-io configuration to use it. It probably wouldn't even count
> as a security issue, because it does not work around the already
> granted access privileges.
> All these _standard_ configuration options and mechanisms are the root
> cause, but it is neither a new issue nor a Polaris issue nor can
> Polaris do anything to prevent this from ever happening.
>
> A different previously mentioned aspect that I'd like to highlight:
> There is currently no way to gain insights into *who* did *what*
> against the object store, because the object storage credentials that
> are currently vended/issued by Polaris are _not_ linked to a
> user/client identity. AssumeRoleWithWebIdentity seems to enable
> getting this very valuable information, which can be necessary for
> certain legal/governance requirements.
>
> Looking a bit around in the ecosystem (disclaimer: I haven't looked at
> the nitty gritty details), Trino and Hive already support S3 via
> AssumeRoleWithWebIdentity.
>
> As already mentioned in the previous messages, "trust" is the key for
> the STS/AssumeRoleWithWebIdentity scenario.
> The STS service should ideally require a token issued for Polaris on
> behalf of the user ("delegation"). That's a legit scenario and a very
> valuable feature for Polaris, in addition to the request to support
> AssumeRoleWithWebIdentity. (Mentioned in the previous messages.)
>
> The "normal" workflow would be: client -> Polaris -> Polaris gets
> down-scoped credentials from STS -> Polaris returns (potentially
> down-scoped) credentials to the client.
> Again, a client can also call STS directly to get S3 credentials via
> AssumeRoleWithWebIdentity, and work with *any* tool directly against
> S3.
> There is nothing that can prevent this from happening, as long as the
> STS service is accessible and STS accepts the JWT ("trust").
>
> Technically and assuming that the user has the necessary access
> privileges, all operations, even those that effectively bypass
> validations and checks in Polaris, are possible.
> This is, again, not a problem with AssumeRoleWithWebIdentity, but a
> general and not a Polaris specific problem. That the source of truth
> (table/view metadata) is in the object storage contributes to the
> situation. Plus: that the client generates the path names and the
> contents of all meta information.
>
> Polaris does govern access to the meta information about namespaces
> and tables and views. For the latter it is, beside some additional
> properties, just the pointer to the table/view metadata. If a user has
> the necessary privileges, overwriting table/view metadata is possible.
>
> What I read between the lines in some of the previous messages is an
> ask for a "perfect security model". Due to the API and the
> architecture we have to work with, it is IMO not possible. It might
> help to generally restrict access to the object store to totally
> controlled systems in an isolated security enclave, which is a very
> different thing.
>
> My proposal is to enable _all_ Polaris users to gain the value add of
> the original proposal, and not push our users towards maintaining hard
> forks for general-purpose functionality:
>
> 1. Continue the AssumeRoleWithWebIdentity proposal, opt-in via a
> catalog or realm level configuration, maybe guarded with a "production
> ready warning", if people really insist
> 2. Add OAuth delegation support to Polaris, starting with the
> AssumeRoleWithWebIdentity use case
>    2.a) Polaris tokens: re-issue the token with a specific claim trusted
> by STS
>    2.b) Delegation for "proper" IdPs
>
> Yes, 1 without 2 might be a concern.
>
> How does that sound?
>
> Robert
>
> On Sat, Dec 6, 2025 at 1:46 AM Adnan Hemani via dev
> <de...@polaris.apache.org> wrote:
> >
> > Hi all,
> >
> > I can get behind Yufei's idea - I don't see any security concerns with
> it.
> > In this case, the admin is explicitly electing (at an application level)
> to
> > outsource their AuthZ to an external system and there still remains a
> > single source of truth of which principals are allowed to access what. To
> > put it in explicit terms, Polaris authorization should be *fully
> disabled*
> > for any catalog that elects this model - and we *must* enforce disabling
> > Polaris' own authorization checks to prevent users being misled and/or
> > confused by this feature.
> >
> > But I also agree with Laurent's analysis of where the OIDC/JWT world is
> > going today - and I think we should invest in supporting that workflow
> in a
> > similar manner to what Laurent is expressing. That model will allow users
> > to still use Polaris authorization *AND* AssumeRoleWithWebIdentity APIs
> for
> > their on-prem (or even cloud) platforms in a secure, non-confusing
> manner -
> > which is the ideal scenario to me.
> >
> > However for RJ's use case, the question remains, will this on-prem "STS"
> > system be able to integrate with the External Policy Decision Point (PDP
> > <
> https://polaris.apache.org/in-dev/unreleased/managing-security/external-pdp/
> >)
> > model?
> > I have no background in what system is being used, so it's hard for me to
> > tell.
> >
> > Best,
> > Adnan Hemani
> >
> > On Fri, Dec 5, 2025 at 10:09 AM Yufei Gu <fl...@gmail.com> wrote:
> >
> > > Hi everyone,
> > >
> > > To add one more perspective here, this is not only a security concern
> but
> > > also a major usability and mental-model concern for Polaris users.
> Here is
> > > an example:
> > >
> > >    - Alice has read + write privileges at the STS/storage layer.
> > >    - In Polaris, Alice is intentionally granted read-only access to a
> > > table.
> > >
> > > Under the proposed model, because the user JWT is forwarded directly to
> > > STS, Alice can simply bypass Polaris and call STS directly. STS then
> > > (correctly, from its own viewpoint) grants her write permission, since
> it
> > > has no awareness of Polaris’ table-level authorization.
> > >
> > > This results in *privilege escalation: *Polaris says “read-only,” but
> Alice
> > > can still perform writes by going around Polaris.
> > >
> > > Beyond the security implications, this is extremely confusing for
> admins
> > > and users. Polaris presents one authoritative access decision, but the
> > > underlying storage enforces another. Users cannot reasonably be
> expected to
> > > understand why they can “write when Polaris says they can’t,”, and
> > > why audit logs and enforcement don’t align.
> > >
> > > *A possible path forward:* one direction we can take, which keeps your
> use
> > > case viable while also preserving Polaris’ governance guarantees, is to
> > > introduce an OPA-style authorization module:
> > >
> > >    - Polaris can delegate authZ decisions to a remote system (e.g.,
> OPA,
> > >    your existing STS-side policy engine, etc).
> > >    - Credential vending still occurs in Polaris, but only after the
> > >    delegated AuthZ result has been evaluated.
> > >
> > > This approach allows deployments like yours to centralize their
> existing
> > > rules in the external system, while also ensuring:
> > >
> > >    - No privilege escalation occurs by bypassing Polaris
> > >    - Credential vending always honors Polaris’ (or delegated) AuthZ
> > > decision
> > >    - We avoid creating two conflicting interpretations of the same
> user’s
> > >    permissions
> > >
> > > This feels like a direction that could satisfy both RJ’s on-prem
> scenario
> > > and the broader community’s expectations around governance semantics.
> > >
> > > Happy to continue iterating on how such a module would be wired in, but
> > > conceptually it gives us a clean separation: AuthZ may be delegated,
> but
> > > enforcement remains consistent and predictable through Polaris.
> > >
> > > Yufei
> > >
> > >
> > > On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <di...@apache.org>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > I guess my comment in GH regarding Polaris-owned tokens was not
> precise
> > > > enough. Apologies for the confusion and I'll try to clarify below.
> > > >
> > > > As far as I understand, in RJ's use case, Polaris is actually
> expected to
> > > > pass the user's auth token from Polaris API to STS. Arguably, it is a
> > > very
> > > > specific use case and may be applicable only to some very specific
> > > on-prem
> > > > environments (please correct me if I misinterpret that), but I
> believe it
> > > > is fine to support.
> > > >
> > > > In that use case Polaris acts as a Resource server from the OAuth2
> > > > perspective and no extra auth tokens are necessary.
> > > >
> > > > Outside of RJ's use case (and PR), I do not really object to
> supporting
> > > > OAuth2 token exchange in Polaris. It could be a valuable feature, but
> > > from
> > > > my POV it is a totally separate feature (not related to [3170])
> > > >
> > > > [3170] https://github.com/apache/polaris/pull/3170
> > > >
> > > > Cheers,
> > > > Dmitri.
> > > >
> > > > On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> > > > dev@polaris.apache.org> wrote:
> > > >
> > > > > UNCLASSIFIED / NON CLASSIFIÉ
> > > > >
> > > > > Hi Laurent, Thanks for your questions!
> > > > >
> > > > > > I'm a big fan of those token exchange scenarios which avoid to
> create
> > > > > long lived secrets to interact between systems, and rely instead of
> > > > setting
> > > > > trust mechanisms between them.
> > > > >
> > > > > We are too, which is why can’t go with the standard
> > > > > access-key-id/access-key-secret setup that a regular AWS S3 system
> > > uses.
> > > > By
> > > > > passing along the short lived JWT token of the user to the STS, we
> can
> > > > get
> > > > > a new, short-lived, access-key-id/access-key-secret for our
> on-prem S3
> > > > > buckets with every request. Having one generic service account
> doing
> > > all
> > > > > the operations on the S3 for all our users is a security risk for
> us.
> > > > >
> > > > >
> > > > > > My main question is that if the authorization token sent to
> polaris
> > > is
> > > > > passed as-is to the STS server, what happens if the client directly
> > > sends
> > > > > the token to the STS system directly?
> > > > >
> > > > > For our on-prem S3/STS setup we have no issue with users accessing
> STS
> > > > > System directly. It is normal for users to be able to access the
> > > buckets.
> > > > > The STS checks the permissions users have on the buckets they’re
> > > > > requesting, so there is no way for a user to gain more privileges
> to
> > > > > potentially read/write a bucket  they don’t get access to through
> > > > polaris,
> > > > > by going directly to the STS. If anything if a user going directly
> to
> > > the
> > > > > STS they’d get MORE permissions, which is totally fine and
> completely
> > > > > reasonable.
> > > > >
> > > > >
> > > > > > You probably have thought of that scenario already so you may
> have
> > > some
> > > > > contingency system in place or maybe it's not a actual issue in
> your
> > > > > deployment situation?
> > > > >
> > > > > We heavily audit what the users are doing. Polaris will log “this
> user
> > > is
> > > > > requesting credentials for this table”, the STS logs “this user is
> > > > > requesting credentials for this bucket” and our on-prem S3 logs
> “this
> > > > user
> > > > > did <thing> to this bucket.” There is no concern from us over
> someone
> > > > > potentially skirting privileges.
> > > > >
> > > > >
> > > > > > One idea I had to prevent user to bypass Polaris authorization
> system
> > > > > would be for Polaris itself to issue its own JWT tokens, and when
> > > > accessing
> > > > > storage, Polaris would do an internal exchange where claims from
> the
> > > > > inbound token (received by Polaris) would be copied over to the
> > > outbound
> > > > > token (sent to STS), after proper validation of course. Is it a
> > > scenario
> > > > > you have considered by any chance?
> > > > >
> > > > > From what I gather, Polaris has no intentions on being in the
> business
> > > of
> > > > > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have
> no
> > > > issue
> > > > > with users going around Polaris to access the STS and S3, this is
> by
> > > > > design. There is no guarantee that this feature will work for
> AWS/Minio
> > > > > [2], it will enable Polaris to support on-prem S3 use cases like
> [3],
> > > > > making it more widely available for everyone.
> > > > >
> > > > > [1]
> > > https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > > > > [2]
> > > https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > > > > [3] https://github.com/apache/polaris/issues/3038
> > > > >
> > > > > Thank you for your feedback and please let me know if there’s any
> more
> > > > > questions!
> > > > >
> > > > > Thanks,
> > > > > R.J.
> > > > >
> > > >
> > >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Robert Stupp <sn...@snazy.de>]

Hi all,

Seeing such a big engagement is amazing! Having so many people care
about security and user oriented features is absolutely awesome for
the Polaris project!

Truth to be told: security is relative and needs to be considered in a
holistic way. Security can never be 100% perfect (it's also software).
It is totally valid and actually required to uphold strong and strict
security standards, wherever possible and wherever it makes sense.
This is what I fully support!
On the other hand, it doesn't make much sense to lock the door if you
can walk into the house through the open garage.

Full disclosure: I'll deviate into very related aspects and pieces to
cover the "whole picture".

From the above messages, it feels like there is some confusion about
what Polaris can fully control/enforce and what Polaris can only
"recommend" due to the API and architecture and libraries we and our
users have to use. More on this below.

The term "privilege escalation" in the previous messages caught my attention.
A "privilege escalation" is a situation when a user can gain *more* or
*higher* access privileges (against the same or a different system).
In this AssumeRoleWithWebIdentity case, the user already has the
privileges. Nothing that's been described gives a user *more* or
*higher* access privileges.
Polaris returns (potentially down-scoped) credentials leveraging IAM
policies, but those can already be overridden by users with different
credentials. This allows a (bad?) client, for example, to write data
to arbitrary locations, effectively bypassing the restrictions of
down-scoped credentials to enforce table locations. But even that can
already be worked around using standard configuration options (think:
using write.folder-storage.path or write.object-storage.path).
For a "smoother experience", it's also possible for users to extend
the S3FileIO to automatically retrieve and refresh the object storage
credentials directly via STS/AssumeRoleWithWebIdentity - just override
the file-io configuration to use it. It probably wouldn't even count
as a security issue, because it does not work around the already
granted access privileges.
All these _standard_ configuration options and mechanisms are the root
cause, but it is neither a new issue nor a Polaris issue nor can
Polaris do anything to prevent this from ever happening.

A different previously mentioned aspect that I'd like to highlight:
There is currently no way to gain insights into *who* did *what*
against the object store, because the object storage credentials that
are currently vended/issued by Polaris are _not_ linked to a
user/client identity. AssumeRoleWithWebIdentity seems to enable
getting this very valuable information, which can be necessary for
certain legal/governance requirements.

Looking a bit around in the ecosystem (disclaimer: I haven't looked at
the nitty gritty details), Trino and Hive already support S3 via
AssumeRoleWithWebIdentity.

As already mentioned in the previous messages, "trust" is the key for
the STS/AssumeRoleWithWebIdentity scenario.
The STS service should ideally require a token issued for Polaris on
behalf of the user ("delegation"). That's a legit scenario and a very
valuable feature for Polaris, in addition to the request to support
AssumeRoleWithWebIdentity. (Mentioned in the previous messages.)

The "normal" workflow would be: client -> Polaris -> Polaris gets
down-scoped credentials from STS -> Polaris returns (potentially
down-scoped) credentials to the client.
Again, a client can also call STS directly to get S3 credentials via
AssumeRoleWithWebIdentity, and work with *any* tool directly against
S3.
There is nothing that can prevent this from happening, as long as the
STS service is accessible and STS accepts the JWT ("trust").

Technically and assuming that the user has the necessary access
privileges, all operations, even those that effectively bypass
validations and checks in Polaris, are possible.
This is, again, not a problem with AssumeRoleWithWebIdentity, but a
general and not a Polaris specific problem. That the source of truth
(table/view metadata) is in the object storage contributes to the
situation. Plus: that the client generates the path names and the
contents of all meta information.

Polaris does govern access to the meta information about namespaces
and tables and views. For the latter it is, beside some additional
properties, just the pointer to the table/view metadata. If a user has
the necessary privileges, overwriting table/view metadata is possible.

What I read between the lines in some of the previous messages is an
ask for a "perfect security model". Due to the API and the
architecture we have to work with, it is IMO not possible. It might
help to generally restrict access to the object store to totally
controlled systems in an isolated security enclave, which is a very
different thing.

My proposal is to enable _all_ Polaris users to gain the value add of
the original proposal, and not push our users towards maintaining hard
forks for general-purpose functionality:

1. Continue the AssumeRoleWithWebIdentity proposal, opt-in via a
catalog or realm level configuration, maybe guarded with a "production
ready warning", if people really insist
2. Add OAuth delegation support to Polaris, starting with the
AssumeRoleWithWebIdentity use case
   2.a) Polaris tokens: re-issue the token with a specific claim trusted by STS
   2.b) Delegation for "proper" IdPs

Yes, 1 without 2 might be a concern.

How does that sound?

Robert

On Sat, Dec 6, 2025 at 1:46 AM Adnan Hemani via dev
<de...@polaris.apache.org> wrote:
>
> Hi all,
>
> I can get behind Yufei's idea - I don't see any security concerns with it.
> In this case, the admin is explicitly electing (at an application level) to
> outsource their AuthZ to an external system and there still remains a
> single source of truth of which principals are allowed to access what. To
> put it in explicit terms, Polaris authorization should be *fully disabled*
> for any catalog that elects this model - and we *must* enforce disabling
> Polaris' own authorization checks to prevent users being misled and/or
> confused by this feature.
>
> But I also agree with Laurent's analysis of where the OIDC/JWT world is
> going today - and I think we should invest in supporting that workflow in a
> similar manner to what Laurent is expressing. That model will allow users
> to still use Polaris authorization *AND* AssumeRoleWithWebIdentity APIs for
> their on-prem (or even cloud) platforms in a secure, non-confusing manner -
> which is the ideal scenario to me.
>
> However for RJ's use case, the question remains, will this on-prem "STS"
> system be able to integrate with the External Policy Decision Point (PDP
> <https://polaris.apache.org/in-dev/unreleased/managing-security/external-pdp/>)
> model?
> I have no background in what system is being used, so it's hard for me to
> tell.
>
> Best,
> Adnan Hemani
>
> On Fri, Dec 5, 2025 at 10:09 AM Yufei Gu <fl...@gmail.com> wrote:
>
> > Hi everyone,
> >
> > To add one more perspective here, this is not only a security concern but
> > also a major usability and mental-model concern for Polaris users. Here is
> > an example:
> >
> >    - Alice has read + write privileges at the STS/storage layer.
> >    - In Polaris, Alice is intentionally granted read-only access to a
> > table.
> >
> > Under the proposed model, because the user JWT is forwarded directly to
> > STS, Alice can simply bypass Polaris and call STS directly. STS then
> > (correctly, from its own viewpoint) grants her write permission, since it
> > has no awareness of Polaris’ table-level authorization.
> >
> > This results in *privilege escalation: *Polaris says “read-only,” but Alice
> > can still perform writes by going around Polaris.
> >
> > Beyond the security implications, this is extremely confusing for admins
> > and users. Polaris presents one authoritative access decision, but the
> > underlying storage enforces another. Users cannot reasonably be expected to
> > understand why they can “write when Polaris says they can’t,”, and
> > why audit logs and enforcement don’t align.
> >
> > *A possible path forward:* one direction we can take, which keeps your use
> > case viable while also preserving Polaris’ governance guarantees, is to
> > introduce an OPA-style authorization module:
> >
> >    - Polaris can delegate authZ decisions to a remote system (e.g., OPA,
> >    your existing STS-side policy engine, etc).
> >    - Credential vending still occurs in Polaris, but only after the
> >    delegated AuthZ result has been evaluated.
> >
> > This approach allows deployments like yours to centralize their existing
> > rules in the external system, while also ensuring:
> >
> >    - No privilege escalation occurs by bypassing Polaris
> >    - Credential vending always honors Polaris’ (or delegated) AuthZ
> > decision
> >    - We avoid creating two conflicting interpretations of the same user’s
> >    permissions
> >
> > This feels like a direction that could satisfy both RJ’s on-prem scenario
> > and the broader community’s expectations around governance semantics.
> >
> > Happy to continue iterating on how such a module would be wired in, but
> > conceptually it gives us a clean separation: AuthZ may be delegated, but
> > enforcement remains consistent and predictable through Polaris.
> >
> > Yufei
> >
> >
> > On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> >
> > > Hi All,
> > >
> > > I guess my comment in GH regarding Polaris-owned tokens was not precise
> > > enough. Apologies for the confusion and I'll try to clarify below.
> > >
> > > As far as I understand, in RJ's use case, Polaris is actually expected to
> > > pass the user's auth token from Polaris API to STS. Arguably, it is a
> > very
> > > specific use case and may be applicable only to some very specific
> > on-prem
> > > environments (please correct me if I misinterpret that), but I believe it
> > > is fine to support.
> > >
> > > In that use case Polaris acts as a Resource server from the OAuth2
> > > perspective and no extra auth tokens are necessary.
> > >
> > > Outside of RJ's use case (and PR), I do not really object to supporting
> > > OAuth2 token exchange in Polaris. It could be a valuable feature, but
> > from
> > > my POV it is a totally separate feature (not related to [3170])
> > >
> > > [3170] https://github.com/apache/polaris/pull/3170
> > >
> > > Cheers,
> > > Dmitri.
> > >
> > > On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> > > dev@polaris.apache.org> wrote:
> > >
> > > > UNCLASSIFIED / NON CLASSIFIÉ
> > > >
> > > > Hi Laurent, Thanks for your questions!
> > > >
> > > > > I'm a big fan of those token exchange scenarios which avoid to create
> > > > long lived secrets to interact between systems, and rely instead of
> > > setting
> > > > trust mechanisms between them.
> > > >
> > > > We are too, which is why can’t go with the standard
> > > > access-key-id/access-key-secret setup that a regular AWS S3 system
> > uses.
> > > By
> > > > passing along the short lived JWT token of the user to the STS, we can
> > > get
> > > > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > > > buckets with every request. Having one generic service account doing
> > all
> > > > the operations on the S3 for all our users is a security risk for us.
> > > >
> > > >
> > > > > My main question is that if the authorization token sent to polaris
> > is
> > > > passed as-is to the STS server, what happens if the client directly
> > sends
> > > > the token to the STS system directly?
> > > >
> > > > For our on-prem S3/STS setup we have no issue with users accessing STS
> > > > System directly. It is normal for users to be able to access the
> > buckets.
> > > > The STS checks the permissions users have on the buckets they’re
> > > > requesting, so there is no way for a user to gain more privileges to
> > > > potentially read/write a bucket  they don’t get access to through
> > > polaris,
> > > > by going directly to the STS. If anything if a user going directly to
> > the
> > > > STS they’d get MORE permissions, which is totally fine and completely
> > > > reasonable.
> > > >
> > > >
> > > > > You probably have thought of that scenario already so you may have
> > some
> > > > contingency system in place or maybe it's not a actual issue in your
> > > > deployment situation?
> > > >
> > > > We heavily audit what the users are doing. Polaris will log “this user
> > is
> > > > requesting credentials for this table”, the STS logs “this user is
> > > > requesting credentials for this bucket” and our on-prem S3 logs “this
> > > user
> > > > did <thing> to this bucket.” There is no concern from us over someone
> > > > potentially skirting privileges.
> > > >
> > > >
> > > > > One idea I had to prevent user to bypass Polaris authorization system
> > > > would be for Polaris itself to issue its own JWT tokens, and when
> > > accessing
> > > > storage, Polaris would do an internal exchange where claims from the
> > > > inbound token (received by Polaris) would be copied over to the
> > outbound
> > > > token (sent to STS), after proper validation of course. Is it a
> > scenario
> > > > you have considered by any chance?
> > > >
> > > > From what I gather, Polaris has no intentions on being in the business
> > of
> > > > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> > > issue
> > > > with users going around Polaris to access the STS and S3, this is by
> > > > design. There is no guarantee that this feature will work for AWS/Minio
> > > > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > > > making it more widely available for everyone.
> > > >
> > > > [1]
> > https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > > > [2]
> > https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > > > [3] https://github.com/apache/polaris/issues/3038
> > > >
> > > > Thank you for your feedback and please let me know if there’s any more
> > > > questions!
> > > >
> > > > Thanks,
> > > > R.J.
> > > >
> > >
> >

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Adnan Hemani via dev <de...@polaris.apache.org>]

Hi all,

I can get behind Yufei's idea - I don't see any security concerns with it.
In this case, the admin is explicitly electing (at an application level) to
outsource their AuthZ to an external system and there still remains a
single source of truth of which principals are allowed to access what. To
put it in explicit terms, Polaris authorization should be *fully disabled*
for any catalog that elects this model - and we *must* enforce disabling
Polaris' own authorization checks to prevent users being misled and/or
confused by this feature.

But I also agree with Laurent's analysis of where the OIDC/JWT world is
going today - and I think we should invest in supporting that workflow in a
similar manner to what Laurent is expressing. That model will allow users
to still use Polaris authorization *AND* AssumeRoleWithWebIdentity APIs for
their on-prem (or even cloud) platforms in a secure, non-confusing manner -
which is the ideal scenario to me.

However for RJ's use case, the question remains, will this on-prem "STS"
system be able to integrate with the External Policy Decision Point (PDP
<https://polaris.apache.org/in-dev/unreleased/managing-security/external-pdp/>)
model?
I have no background in what system is being used, so it's hard for me to
tell.

Best,
Adnan Hemani

On Fri, Dec 5, 2025 at 10:09 AM Yufei Gu <fl...@gmail.com> wrote:

> Hi everyone,
>
> To add one more perspective here, this is not only a security concern but
> also a major usability and mental-model concern for Polaris users. Here is
> an example:
>
>    - Alice has read + write privileges at the STS/storage layer.
>    - In Polaris, Alice is intentionally granted read-only access to a
> table.
>
> Under the proposed model, because the user JWT is forwarded directly to
> STS, Alice can simply bypass Polaris and call STS directly. STS then
> (correctly, from its own viewpoint) grants her write permission, since it
> has no awareness of Polaris’ table-level authorization.
>
> This results in *privilege escalation: *Polaris says “read-only,” but Alice
> can still perform writes by going around Polaris.
>
> Beyond the security implications, this is extremely confusing for admins
> and users. Polaris presents one authoritative access decision, but the
> underlying storage enforces another. Users cannot reasonably be expected to
> understand why they can “write when Polaris says they can’t,”, and
> why audit logs and enforcement don’t align.
>
> *A possible path forward:* one direction we can take, which keeps your use
> case viable while also preserving Polaris’ governance guarantees, is to
> introduce an OPA-style authorization module:
>
>    - Polaris can delegate authZ decisions to a remote system (e.g., OPA,
>    your existing STS-side policy engine, etc).
>    - Credential vending still occurs in Polaris, but only after the
>    delegated AuthZ result has been evaluated.
>
> This approach allows deployments like yours to centralize their existing
> rules in the external system, while also ensuring:
>
>    - No privilege escalation occurs by bypassing Polaris
>    - Credential vending always honors Polaris’ (or delegated) AuthZ
> decision
>    - We avoid creating two conflicting interpretations of the same user’s
>    permissions
>
> This feels like a direction that could satisfy both RJ’s on-prem scenario
> and the broader community’s expectations around governance semantics.
>
> Happy to continue iterating on how such a module would be wired in, but
> conceptually it gives us a clean separation: AuthZ may be delegated, but
> enforcement remains consistent and predictable through Polaris.
>
> Yufei
>
>
> On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
> > Hi All,
> >
> > I guess my comment in GH regarding Polaris-owned tokens was not precise
> > enough. Apologies for the confusion and I'll try to clarify below.
> >
> > As far as I understand, in RJ's use case, Polaris is actually expected to
> > pass the user's auth token from Polaris API to STS. Arguably, it is a
> very
> > specific use case and may be applicable only to some very specific
> on-prem
> > environments (please correct me if I misinterpret that), but I believe it
> > is fine to support.
> >
> > In that use case Polaris acts as a Resource server from the OAuth2
> > perspective and no extra auth tokens are necessary.
> >
> > Outside of RJ's use case (and PR), I do not really object to supporting
> > OAuth2 token exchange in Polaris. It could be a valuable feature, but
> from
> > my POV it is a totally separate feature (not related to [3170])
> >
> > [3170] https://github.com/apache/polaris/pull/3170
> >
> > Cheers,
> > Dmitri.
> >
> > On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> > dev@polaris.apache.org> wrote:
> >
> > > UNCLASSIFIED / NON CLASSIFIÉ
> > >
> > > Hi Laurent, Thanks for your questions!
> > >
> > > > I'm a big fan of those token exchange scenarios which avoid to create
> > > long lived secrets to interact between systems, and rely instead of
> > setting
> > > trust mechanisms between them.
> > >
> > > We are too, which is why can’t go with the standard
> > > access-key-id/access-key-secret setup that a regular AWS S3 system
> uses.
> > By
> > > passing along the short lived JWT token of the user to the STS, we can
> > get
> > > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > > buckets with every request. Having one generic service account doing
> all
> > > the operations on the S3 for all our users is a security risk for us.
> > >
> > >
> > > > My main question is that if the authorization token sent to polaris
> is
> > > passed as-is to the STS server, what happens if the client directly
> sends
> > > the token to the STS system directly?
> > >
> > > For our on-prem S3/STS setup we have no issue with users accessing STS
> > > System directly. It is normal for users to be able to access the
> buckets.
> > > The STS checks the permissions users have on the buckets they’re
> > > requesting, so there is no way for a user to gain more privileges to
> > > potentially read/write a bucket  they don’t get access to through
> > polaris,
> > > by going directly to the STS. If anything if a user going directly to
> the
> > > STS they’d get MORE permissions, which is totally fine and completely
> > > reasonable.
> > >
> > >
> > > > You probably have thought of that scenario already so you may have
> some
> > > contingency system in place or maybe it's not a actual issue in your
> > > deployment situation?
> > >
> > > We heavily audit what the users are doing. Polaris will log “this user
> is
> > > requesting credentials for this table”, the STS logs “this user is
> > > requesting credentials for this bucket” and our on-prem S3 logs “this
> > user
> > > did <thing> to this bucket.” There is no concern from us over someone
> > > potentially skirting privileges.
> > >
> > >
> > > > One idea I had to prevent user to bypass Polaris authorization system
> > > would be for Polaris itself to issue its own JWT tokens, and when
> > accessing
> > > storage, Polaris would do an internal exchange where claims from the
> > > inbound token (received by Polaris) would be copied over to the
> outbound
> > > token (sent to STS), after proper validation of course. Is it a
> scenario
> > > you have considered by any chance?
> > >
> > > From what I gather, Polaris has no intentions on being in the business
> of
> > > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> > issue
> > > with users going around Polaris to access the STS and S3, this is by
> > > design. There is no guarantee that this feature will work for AWS/Minio
> > > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > > making it more widely available for everyone.
> > >
> > > [1]
> https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > > [2]
> https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > > [3] https://github.com/apache/polaris/issues/3038
> > >
> > > Thank you for your feedback and please let me know if there’s any more
> > > questions!
> > >
> > > Thanks,
> > > R.J.
> > >
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Yufei Gu <fl...@gmail.com>]

Hi everyone,

To add one more perspective here, this is not only a security concern but
also a major usability and mental-model concern for Polaris users. Here is
an example:

   - Alice has read + write privileges at the STS/storage layer.
   - In Polaris, Alice is intentionally granted read-only access to a table.

Under the proposed model, because the user JWT is forwarded directly to
STS, Alice can simply bypass Polaris and call STS directly. STS then
(correctly, from its own viewpoint) grants her write permission, since it
has no awareness of Polaris’ table-level authorization.

This results in *privilege escalation: *Polaris says “read-only,” but Alice
can still perform writes by going around Polaris.

Beyond the security implications, this is extremely confusing for admins
and users. Polaris presents one authoritative access decision, but the
underlying storage enforces another. Users cannot reasonably be expected to
understand why they can “write when Polaris says they can’t,”, and
why audit logs and enforcement don’t align.

*A possible path forward:* one direction we can take, which keeps your use
case viable while also preserving Polaris’ governance guarantees, is to
introduce an OPA-style authorization module:

   - Polaris can delegate authZ decisions to a remote system (e.g., OPA,
   your existing STS-side policy engine, etc).
   - Credential vending still occurs in Polaris, but only after the
   delegated AuthZ result has been evaluated.

This approach allows deployments like yours to centralize their existing
rules in the external system, while also ensuring:

   - No privilege escalation occurs by bypassing Polaris
   - Credential vending always honors Polaris’ (or delegated) AuthZ decision
   - We avoid creating two conflicting interpretations of the same user’s
   permissions

This feels like a direction that could satisfy both RJ’s on-prem scenario
and the broader community’s expectations around governance semantics.

Happy to continue iterating on how such a module would be wired in, but
conceptually it gives us a clean separation: AuthZ may be delegated, but
enforcement remains consistent and predictable through Polaris.

Yufei


On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <di...@apache.org> wrote:

> Hi All,
>
> I guess my comment in GH regarding Polaris-owned tokens was not precise
> enough. Apologies for the confusion and I'll try to clarify below.
>
> As far as I understand, in RJ's use case, Polaris is actually expected to
> pass the user's auth token from Polaris API to STS. Arguably, it is a very
> specific use case and may be applicable only to some very specific on-prem
> environments (please correct me if I misinterpret that), but I believe it
> is fine to support.
>
> In that use case Polaris acts as a Resource server from the OAuth2
> perspective and no extra auth tokens are necessary.
>
> Outside of RJ's use case (and PR), I do not really object to supporting
> OAuth2 token exchange in Polaris. It could be a valuable feature, but from
> my POV it is a totally separate feature (not related to [3170])
>
> [3170] https://github.com/apache/polaris/pull/3170
>
> Cheers,
> Dmitri.
>
> On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> dev@polaris.apache.org> wrote:
>
> > UNCLASSIFIED / NON CLASSIFIÉ
> >
> > Hi Laurent, Thanks for your questions!
> >
> > > I'm a big fan of those token exchange scenarios which avoid to create
> > long lived secrets to interact between systems, and rely instead of
> setting
> > trust mechanisms between them.
> >
> > We are too, which is why can’t go with the standard
> > access-key-id/access-key-secret setup that a regular AWS S3 system uses.
> By
> > passing along the short lived JWT token of the user to the STS, we can
> get
> > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > buckets with every request. Having one generic service account doing all
> > the operations on the S3 for all our users is a security risk for us.
> >
> >
> > > My main question is that if the authorization token sent to polaris is
> > passed as-is to the STS server, what happens if the client directly sends
> > the token to the STS system directly?
> >
> > For our on-prem S3/STS setup we have no issue with users accessing STS
> > System directly. It is normal for users to be able to access the buckets.
> > The STS checks the permissions users have on the buckets they’re
> > requesting, so there is no way for a user to gain more privileges to
> > potentially read/write a bucket  they don’t get access to through
> polaris,
> > by going directly to the STS. If anything if a user going directly to the
> > STS they’d get MORE permissions, which is totally fine and completely
> > reasonable.
> >
> >
> > > You probably have thought of that scenario already so you may have some
> > contingency system in place or maybe it's not a actual issue in your
> > deployment situation?
> >
> > We heavily audit what the users are doing. Polaris will log “this user is
> > requesting credentials for this table”, the STS logs “this user is
> > requesting credentials for this bucket” and our on-prem S3 logs “this
> user
> > did <thing> to this bucket.” There is no concern from us over someone
> > potentially skirting privileges.
> >
> >
> > > One idea I had to prevent user to bypass Polaris authorization system
> > would be for Polaris itself to issue its own JWT tokens, and when
> accessing
> > storage, Polaris would do an internal exchange where claims from the
> > inbound token (received by Polaris) would be copied over to the outbound
> > token (sent to STS), after proper validation of course. Is it a scenario
> > you have considered by any chance?
> >
> > From what I gather, Polaris has no intentions on being in the business of
> > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> issue
> > with users going around Polaris to access the STS and S3, this is by
> > design. There is no guarantee that this feature will work for AWS/Minio
> > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > making it more widely available for everyone.
> >
> > [1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > [2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > [3] https://github.com/apache/polaris/issues/3038
> >
> > Thank you for your feedback and please let me know if there’s any more
> > questions!
> >
> > Thanks,
> > R.J.
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Dmitri Bourlatchkov <di...@apache.org>]

Hi All,

I guess my comment in GH regarding Polaris-owned tokens was not precise
enough. Apologies for the confusion and I'll try to clarify below.

As far as I understand, in RJ's use case, Polaris is actually expected to
pass the user's auth token from Polaris API to STS. Arguably, it is a very
specific use case and may be applicable only to some very specific on-prem
environments (please correct me if I misinterpret that), but I believe it
is fine to support.

In that use case Polaris acts as a Resource server from the OAuth2
perspective and no extra auth tokens are necessary.

Outside of RJ's use case (and PR), I do not really object to supporting
OAuth2 token exchange in Polaris. It could be a valuable feature, but from
my POV it is a totally separate feature (not related to [3170])

[3170] https://github.com/apache/polaris/pull/3170

Cheers,
Dmitri.

On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFIÉ
>
> Hi Laurent, Thanks for your questions!
>
> > I'm a big fan of those token exchange scenarios which avoid to create
> long lived secrets to interact between systems, and rely instead of setting
> trust mechanisms between them.
>
> We are too, which is why can’t go with the standard
> access-key-id/access-key-secret setup that a regular AWS S3 system uses. By
> passing along the short lived JWT token of the user to the STS, we can get
> a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> buckets with every request. Having one generic service account doing all
> the operations on the S3 for all our users is a security risk for us.
>
>
> > My main question is that if the authorization token sent to polaris is
> passed as-is to the STS server, what happens if the client directly sends
> the token to the STS system directly?
>
> For our on-prem S3/STS setup we have no issue with users accessing STS
> System directly. It is normal for users to be able to access the buckets.
> The STS checks the permissions users have on the buckets they’re
> requesting, so there is no way for a user to gain more privileges to
> potentially read/write a bucket  they don’t get access to through polaris,
> by going directly to the STS. If anything if a user going directly to the
> STS they’d get MORE permissions, which is totally fine and completely
> reasonable.
>
>
> > You probably have thought of that scenario already so you may have some
> contingency system in place or maybe it's not a actual issue in your
> deployment situation?
>
> We heavily audit what the users are doing. Polaris will log “this user is
> requesting credentials for this table”, the STS logs “this user is
> requesting credentials for this bucket” and our on-prem S3 logs “this user
> did <thing> to this bucket.” There is no concern from us over someone
> potentially skirting privileges.
>
>
> > One idea I had to prevent user to bypass Polaris authorization system
> would be for Polaris itself to issue its own JWT tokens, and when accessing
> storage, Polaris would do an internal exchange where claims from the
> inbound token (received by Polaris) would be copied over to the outbound
> token (sent to STS), after proper validation of course. Is it a scenario
> you have considered by any chance?
>
> From what I gather, Polaris has no intentions on being in the business of
> issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no issue
> with users going around Polaris to access the STS and S3, this is by
> design. There is no guarantee that this feature will work for AWS/Minio
> [2], it will enable Polaris to support on-prem S3 use cases like [3],
> making it more widely available for everyone.
>
> [1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> [2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> [3] https://github.com/apache/polaris/issues/3038
>
> Thank you for your feedback and please let me know if there’s any more
> questions!
>
> Thanks,
> R.J.
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Dmitri Bourlatchkov <di...@apache.org>]

Hi RJ and Adnan,

I believe Polaris should not be limited to a small / narrow set of use
cases. As users with different application environments join the project,
supporting their use cases is essential for the community to grow.

I believe Adnan's concerns can be resolved by introducing a realm-level
feature flag to control the new behaviour added in RJ's PR.

Deployments that consider this mode of operation risky, can turn it off
(perhaps that should be the default). Deployments that find it valuable,
can enable.

PR [3170] in its current state does not appear as overly invasive to the
codebase, IMHO. A similar approach is probably going to be needed to
address [3196] anyway. Given the relative simplicity of the code change,
plus a feature flag, I believe supporting RJ's use case does not add any
major security risks to Polaris.

WDYT?

[3170] https://github.com/apache/polaris/pull/3170

[3196] https://github.com/apache/polaris/issues/3196

Thanks,
Dmitri.

On Fri, Dec 5, 2025 at 4:01 AM Adnan Hemani via dev <de...@polaris.apache.org>
wrote:

> Hi R.J.,
>
> Thanks for answering these questions.
>
> > If anything if a user going directly to the STS they’d get MORE
> permissions, which is totally fine and completely reasonable.
>
> This is not the security model that Polaris currently uses. And while in
> your use case this statement is acceptable to you, this is *not* broadly
> acceptable. To restate the above statement IOW: it is completely acceptable
> for users to skirt around Polaris authorization results and I completely
> disagree that Polaris should publicly support a feature that enables such
> behavior. I understand if Polaris' authorization is not important to your
> use case, but that is definitely not broadly the case.
>
> On the topic of auditing: if a user can simply bypass Polaris and talk to
> "STS" to start accessing data, I am assuming that your auditing mechanisms
> do not rely on Polaris' logs being all-encompassing of users' accesses of
> data. Given that is true, is it fair to say in your use case, Polaris is
> not much of a governance tool as much as it is just a simplistic metastore?
>
> On a slightly tangential note, I would like to know more as to why you need
> credential vending in Polaris if your users have the ability to mint their
> own credentials from STS? Is what you need able to be achieved by making a
> slight modification to the Iceberg client instead? Can you explain why
> Polaris is the right place for this change?
>
> > it will enable Polaris to support on-prem S3 use cases like [3], making
> it more widely available for everyone
>
> I agree that we should aim to support more use cases - but in a secure way
> that works in a broadly acceptable manner with Polaris' core
> functionalities (such as authorization and access control). I'm definitely
> happy to brainstorm how to make this design more secure and broadly usable
> - either here or on a different thread.
>
> Best,
> Adnan Hemani
>
>
> On Thu, Dec 4, 2025 at 11:22 AM Arsenault, Reginald P. via dev <
> dev@polaris.apache.org> wrote:
>
> > UNCLASSIFIED / NON CLASSIFIÉ
> >
> > Hi Laurent, Thanks for your questions!
> >
> > > I'm a big fan of those token exchange scenarios which avoid to create
> > long lived secrets to interact between systems, and rely instead of
> setting
> > trust mechanisms between them.
> >
> > We are too, which is why can’t go with the standard
> > access-key-id/access-key-secret setup that a regular AWS S3 system uses.
> By
> > passing along the short lived JWT token of the user to the STS, we can
> get
> > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > buckets with every request. Having one generic service account doing all
> > the operations on the S3 for all our users is a security risk for us.
> >
> >
> > > My main question is that if the authorization token sent to polaris is
> > passed as-is to the STS server, what happens if the client directly sends
> > the token to the STS system directly?
> >
> > For our on-prem S3/STS setup we have no issue with users accessing STS
> > System directly. It is normal for users to be able to access the buckets.
> > The STS checks the permissions users have on the buckets they’re
> > requesting, so there is no way for a user to gain more privileges to
> > potentially read/write a bucket  they don’t get access to through
> polaris,
> > by going directly to the STS. If anything if a user going directly to the
> > STS they’d get MORE permissions, which is totally fine and completely
> > reasonable.
> >
> >
> > > You probably have thought of that scenario already so you may have some
> > contingency system in place or maybe it's not a actual issue in your
> > deployment situation?
> >
> > We heavily audit what the users are doing. Polaris will log “this user is
> > requesting credentials for this table”, the STS logs “this user is
> > requesting credentials for this bucket” and our on-prem S3 logs “this
> user
> > did <thing> to this bucket.” There is no concern from us over someone
> > potentially skirting privileges.
> >
> >
> > > One idea I had to prevent user to bypass Polaris authorization system
> > would be for Polaris itself to issue its own JWT tokens, and when
> accessing
> > storage, Polaris would do an internal exchange where claims from the
> > inbound token (received by Polaris) would be copied over to the outbound
> > token (sent to STS), after proper validation of course. Is it a scenario
> > you have considered by any chance?
> >
> > From what I gather, Polaris has no intentions on being in the business of
> > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> issue
> > with users going around Polaris to access the STS and S3, this is by
> > design. There is no guarantee that this feature will work for AWS/Minio
> > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > making it more widely available for everyone.
> >
> > [1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > [2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > [3] https://github.com/apache/polaris/issues/3038
> >
> > Thank you for your feedback and please let me know if there’s any more
> > questions!
> >
> > Thanks,
> > R.J.
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Laurent Goujon <la...@dremio.com>]

I may not be as categorical as Adnan about this being not broadly
acceptable (because security is a relative concept, hence why context is
important), but a major reason behind the development of credential vending
in IRC is for lakehouse administrator to make the catalog the governance
layer in terms of data access/permissions, and not to have to duplicate the
governance effort at the object storage as well. Your design seems to make
the STS layer the main access control layer instead, but you didn't say if
you were planning to leverage Polaris access control layer (or if everybody
has full access to the catalog, and ultimately final access check is done
when trying to access the data).

That said, it doesn't seem there's anything special about the token and the
way Polaris processes it for credentials vending, so why leveraging
credential vending vs having the iceberg rest client calling directly STS
instead? (same question as Adnan)

> One idea I had to prevent user to bypass Polaris authorization system
> would be for Polaris itself to issue its own JWT tokens, and when accessing
> storage, Polaris would do an internal exchange where claims from the
> inbound token (received by Polaris) would be copied over to the outbound
> token (sent to STS), after proper validation of course. Is it a scenario
> you have considered by any chance?
>
> From what I gather, Polaris has no intentions on being in the business of
> issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no issue
> with users going around Polaris to access the STS and S3, this is by
> design. There is no guarantee that this feature will work for AWS/Minio
> [2], it will enable Polaris to support on-prem S3 use cases like [3],
> making it more widely available for everyone.


It may seem over engineered now because it's not a widely adopted pattern,
but I can see things are evolving: you can see many systems introducing
workload identity federation or OIDC/JWT token exchange capabilities these
days, I just noticed that AWS has introduced a new `GetWebIdentityToken`
API to generate a JWT token out of IAM credentials, and at work we have
been working on something similar to introduce zero-trust/credential less
access to remote systems. So I hope I can convince the community to adopt
such a pattern at some point...





On Fri, Dec 5, 2025 at 1:01 AM Adnan Hemani via dev <de...@polaris.apache.org>
wrote:

> Hi R.J.,
>
> Thanks for answering these questions.
>
> > If anything if a user going directly to the STS they’d get MORE
> permissions, which is totally fine and completely reasonable.
>
> This is not the security model that Polaris currently uses. And while in
> your use case this statement is acceptable to you, this is *not* broadly
> acceptable. To restate the above statement IOW: it is completely acceptable
> for users to skirt around Polaris authorization results and I completely
> disagree that Polaris should publicly support a feature that enables such
> behavior. I understand if Polaris' authorization is not important to your
> use case, but that is definitely not broadly the case.
>
> On the topic of auditing: if a user can simply bypass Polaris and talk to
> "STS" to start accessing data, I am assuming that your auditing mechanisms
> do not rely on Polaris' logs being all-encompassing of users' accesses of
> data. Given that is true, is it fair to say in your use case, Polaris is
> not much of a governance tool as much as it is just a simplistic metastore?
>
> On a slightly tangential note, I would like to know more as to why you need
> credential vending in Polaris if your users have the ability to mint their
> own credentials from STS? Is what you need able to be achieved by making a
> slight modification to the Iceberg client instead? Can you explain why
> Polaris is the right place for this change?
>
> > it will enable Polaris to support on-prem S3 use cases like [3], making
> it more widely available for everyone
>
> I agree that we should aim to support more use cases - but in a secure way
> that works in a broadly acceptable manner with Polaris' core
> functionalities (such as authorization and access control). I'm definitely
> happy to brainstorm how to make this design more secure and broadly usable
> - either here or on a different thread.
>
> Best,
> Adnan Hemani
>
>
> On Thu, Dec 4, 2025 at 11:22 AM Arsenault, Reginald P. via dev <
> dev@polaris.apache.org> wrote:
>
> > UNCLASSIFIED / NON CLASSIFIÉ
> >
> > Hi Laurent, Thanks for your questions!
> >
> > > I'm a big fan of those token exchange scenarios which avoid to create
> > long lived secrets to interact between systems, and rely instead of
> setting
> > trust mechanisms between them.
> >
> > We are too, which is why can’t go with the standard
> > access-key-id/access-key-secret setup that a regular AWS S3 system uses.
> By
> > passing along the short lived JWT token of the user to the STS, we can
> get
> > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > buckets with every request. Having one generic service account doing all
> > the operations on the S3 for all our users is a security risk for us.
> >
> >
> > > My main question is that if the authorization token sent to polaris is
> > passed as-is to the STS server, what happens if the client directly sends
> > the token to the STS system directly?
> >
> > For our on-prem S3/STS setup we have no issue with users accessing STS
> > System directly. It is normal for users to be able to access the buckets.
> > The STS checks the permissions users have on the buckets they’re
> > requesting, so there is no way for a user to gain more privileges to
> > potentially read/write a bucket  they don’t get access to through
> polaris,
> > by going directly to the STS. If anything if a user going directly to the
> > STS they’d get MORE permissions, which is totally fine and completely
> > reasonable.
> >
> >
> > > You probably have thought of that scenario already so you may have some
> > contingency system in place or maybe it's not a actual issue in your
> > deployment situation?
> >
> > We heavily audit what the users are doing. Polaris will log “this user is
> > requesting credentials for this table”, the STS logs “this user is
> > requesting credentials for this bucket” and our on-prem S3 logs “this
> user
> > did <thing> to this bucket.” There is no concern from us over someone
> > potentially skirting privileges.
> >
> >
> > > One idea I had to prevent user to bypass Polaris authorization system
> > would be for Polaris itself to issue its own JWT tokens, and when
> accessing
> > storage, Polaris would do an internal exchange where claims from the
> > inbound token (received by Polaris) would be copied over to the outbound
> > token (sent to STS), after proper validation of course. Is it a scenario
> > you have considered by any chance?
> >
> > From what I gather, Polaris has no intentions on being in the business of
> > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> issue
> > with users going around Polaris to access the STS and S3, this is by
> > design. There is no guarantee that this feature will work for AWS/Minio
> > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > making it more widely available for everyone.
> >
> > [1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > [2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > [3] https://github.com/apache/polaris/issues/3038
> >
> > Thank you for your feedback and please let me know if there’s any more
> > questions!
> >
> > Thanks,
> > R.J.
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Adnan Hemani via dev <de...@polaris.apache.org>]

Hi R.J.,

Thanks for answering these questions.

> If anything if a user going directly to the STS they’d get MORE
permissions, which is totally fine and completely reasonable.

This is not the security model that Polaris currently uses. And while in
your use case this statement is acceptable to you, this is *not* broadly
acceptable. To restate the above statement IOW: it is completely acceptable
for users to skirt around Polaris authorization results and I completely
disagree that Polaris should publicly support a feature that enables such
behavior. I understand if Polaris' authorization is not important to your
use case, but that is definitely not broadly the case.

On the topic of auditing: if a user can simply bypass Polaris and talk to
"STS" to start accessing data, I am assuming that your auditing mechanisms
do not rely on Polaris' logs being all-encompassing of users' accesses of
data. Given that is true, is it fair to say in your use case, Polaris is
not much of a governance tool as much as it is just a simplistic metastore?

On a slightly tangential note, I would like to know more as to why you need
credential vending in Polaris if your users have the ability to mint their
own credentials from STS? Is what you need able to be achieved by making a
slight modification to the Iceberg client instead? Can you explain why
Polaris is the right place for this change?

> it will enable Polaris to support on-prem S3 use cases like [3], making
it more widely available for everyone

I agree that we should aim to support more use cases - but in a secure way
that works in a broadly acceptable manner with Polaris' core
functionalities (such as authorization and access control). I'm definitely
happy to brainstorm how to make this design more secure and broadly usable
- either here or on a different thread.

Best,
Adnan Hemani


On Thu, Dec 4, 2025 at 11:22 AM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFIÉ
>
> Hi Laurent, Thanks for your questions!
>
> > I'm a big fan of those token exchange scenarios which avoid to create
> long lived secrets to interact between systems, and rely instead of setting
> trust mechanisms between them.
>
> We are too, which is why can’t go with the standard
> access-key-id/access-key-secret setup that a regular AWS S3 system uses. By
> passing along the short lived JWT token of the user to the STS, we can get
> a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> buckets with every request. Having one generic service account doing all
> the operations on the S3 for all our users is a security risk for us.
>
>
> > My main question is that if the authorization token sent to polaris is
> passed as-is to the STS server, what happens if the client directly sends
> the token to the STS system directly?
>
> For our on-prem S3/STS setup we have no issue with users accessing STS
> System directly. It is normal for users to be able to access the buckets.
> The STS checks the permissions users have on the buckets they’re
> requesting, so there is no way for a user to gain more privileges to
> potentially read/write a bucket  they don’t get access to through polaris,
> by going directly to the STS. If anything if a user going directly to the
> STS they’d get MORE permissions, which is totally fine and completely
> reasonable.
>
>
> > You probably have thought of that scenario already so you may have some
> contingency system in place or maybe it's not a actual issue in your
> deployment situation?
>
> We heavily audit what the users are doing. Polaris will log “this user is
> requesting credentials for this table”, the STS logs “this user is
> requesting credentials for this bucket” and our on-prem S3 logs “this user
> did <thing> to this bucket.” There is no concern from us over someone
> potentially skirting privileges.
>
>
> > One idea I had to prevent user to bypass Polaris authorization system
> would be for Polaris itself to issue its own JWT tokens, and when accessing
> storage, Polaris would do an internal exchange where claims from the
> inbound token (received by Polaris) would be copied over to the outbound
> token (sent to STS), after proper validation of course. Is it a scenario
> you have considered by any chance?
>
> From what I gather, Polaris has no intentions on being in the business of
> issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no issue
> with users going around Polaris to access the STS and S3, this is by
> design. There is no guarantee that this feature will work for AWS/Minio
> [2], it will enable Polaris to support on-prem S3 use cases like [3],
> making it more widely available for everyone.
>
> [1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> [2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> [3] https://github.com/apache/polaris/issues/3038
>
> Thank you for your feedback and please let me know if there’s any more
> questions!
>
> Thanks,
> R.J.
>

RE: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

["Arsenault, Reginald P. via dev" <de...@polaris.apache.org>]

UNCLASSIFIED / NON CLASSIFIÉ

Hi Laurent, Thanks for your questions!

> I'm a big fan of those token exchange scenarios which avoid to create long lived secrets to interact between systems, and rely instead of setting trust mechanisms between them.

We are too, which is why can’t go with the standard access-key-id/access-key-secret setup that a regular AWS S3 system uses. By passing along the short lived JWT token of the user to the STS, we can get a new, short-lived, access-key-id/access-key-secret for our on-prem S3 buckets with every request. Having one generic service account doing all the operations on the S3 for all our users is a security risk for us.


> My main question is that if the authorization token sent to polaris is passed as-is to the STS server, what happens if the client directly sends the token to the STS system directly?

For our on-prem S3/STS setup we have no issue with users accessing STS System directly. It is normal for users to be able to access the buckets. The STS checks the permissions users have on the buckets they’re requesting, so there is no way for a user to gain more privileges to potentially read/write a bucket  they don’t get access to through polaris, by going directly to the STS. If anything if a user going directly to the STS they’d get MORE permissions, which is totally fine and completely reasonable.


> You probably have thought of that scenario already so you may have some contingency system in place or maybe it's not a actual issue in your deployment situation?

We heavily audit what the users are doing. Polaris will log “this user is requesting credentials for this table”, the STS logs “this user is requesting credentials for this bucket” and our on-prem S3 logs “this user did <thing> to this bucket.” There is no concern from us over someone potentially skirting privileges.


> One idea I had to prevent user to bypass Polaris authorization system would be for Polaris itself to issue its own JWT tokens, and when accessing storage, Polaris would do an internal exchange where claims from the inbound token (received by Polaris) would be copied over to the outbound token (sent to STS), after proper validation of course. Is it a scenario you have considered by any chance?

From what I gather, Polaris has no intentions on being in the business of issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no issue with users going around Polaris to access the STS and S3, this is by design. There is no guarantee that this feature will work for AWS/Minio [2], it will enable Polaris to support on-prem S3 use cases like [3], making it more widely available for everyone.

[1] https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
[2] https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
[3] https://github.com/apache/polaris/issues/3038

Thank you for your feedback and please let me know if there’s any more questions!

Thanks,
R.J.

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Adnan Hemani via dev <de...@polaris.apache.org>]

Hi Laurent,

Great questions - I've raised the same concerns in the PR yesterday, using
the implementation to interpolate the answers to the questions you've
posed:
https://github.com/apache/polaris/pull/3170#pullrequestreview-3532561629.

I will continue responding on the PR - but would prefer we answer these
high-level questions on the ML here.

Best,
Adnan Hemani

On Wed, Dec 3, 2025 at 11:24 AM Laurent Goujon <la...@dremio.com> wrote:

> Hello Reginald,
>
> I have some questions not related to the PRs or code change but regarding
> the general approach, and I hope you don't mind.
>
> I'm a big fan of those token exchange scenarios which avoid to create long
> lived secrets to interact between systems, and rely instead of setting
> trust mechanisms between them. (This is also why I'm looking forward for
> better oauth support in iceberg as well).
>
> My main question is that if the authorization token sent to polaris is
> passed as-is to the STS server, what happens if the client directly sends
> the token to the STS system directly? Wouldn't it be bypassing Polaris
> authorization system and grant the client full access to the storage
> location (to the extend of the IAM policy). You probably have thought of
> that scenario already so you may have some contingency system in place or
> maybe it's not a actual issue in your deployment situation? It woud be
> great if you could elaborate a bit as it is always nice when people
> exchange their best security practices.
>
> One idea I had to prevent user to bypass Polaris authorization system would
> be for Polaris itself to issue its own JWT tokens, and when accessing
> storage, Polaris would do an internal exchange where claims from the
> inbound token (received by Polaris) would be copied over to the outbound
> token (sent to STS), after proper validation of course. Is it a scenario
> you have considered by any chance?
>
> Hope to hear more,
>
> Cheers,
>
> Laurent
>
> On Thu, Nov 20, 2025 at 11:32 AM Arsenault, Reginald P. via dev <
> dev@polaris.apache.org> wrote:
>
> > UNCLASSIFIED / NON CLASSIFI?
> >
> > Hi! As per our findings in [2], we are looking to enable connectivity
> with
> > our on-prem S3 storage appliance, which requires changes to how Polaris
> > connects to S3. Currently, the way Polaris connects to S3 does not work
> for
> > our on-prem S3 instance because of the way it expects to connect to the
> STS
> > server (i.e. without any specific authentication).
> > We propose enhancing the AwsCredentialsStorageIntegration to support a
> > more flexible authorization mechanism:
> >
> >   *   Goal: We want the ability to pass the requesting user's identity
> > (likely a token) to the STS. This would allow the STS to make
> fine-grained
> > decisions on what the user can and cannot do based on their specific
> > identity.
> >   *   Proposed Implementation: We believe this functionality can be
> > achieved by adding an option within AwsCredentialsStorageIntegration [1]
> to
> > connect using the AssumeRoleWithWebIdentity flow.
> > We would greatly appreciate some guidance on the following changes before
> > we begin development:
> >
> >   1.  What is the recommended or existing mechanism within Polaris for
> > capturing and passing along an authenticated user's identity token to the
> > AwsCredentialsStorageIntegration component, so we can pass it to the
> STS? I
> > could be wrong, but I believe the token that gets passed to STS doesn't
> > need to be scoped for the STS, so it could just be whatever token is
> given
> > to Polaris by the user.
> >   2.  How should we best modify the AwsCredentialsStorageIntegration
> class
> > and configuration to be able to handle both the existing AssumeRole use
> > case and the new AssumeRoleWithWebIdentity use case?
> > Any pointers or suggestions on the best place to inject the user token
> and
> > manage the configuration switch would be helpful.
> >
> > [1]
> >
> https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L107
> > [2] https://github.com/apache/polaris/issues/3038
> >
>

Re: [DISCUSS] AWS STS AssumeRoleWithWebIdentity implementation

[Laurent Goujon <la...@dremio.com>]

Hello Reginald,

I have some questions not related to the PRs or code change but regarding
the general approach, and I hope you don't mind.

I'm a big fan of those token exchange scenarios which avoid to create long
lived secrets to interact between systems, and rely instead of setting
trust mechanisms between them. (This is also why I'm looking forward for
better oauth support in iceberg as well).

My main question is that if the authorization token sent to polaris is
passed as-is to the STS server, what happens if the client directly sends
the token to the STS system directly? Wouldn't it be bypassing Polaris
authorization system and grant the client full access to the storage
location (to the extend of the IAM policy). You probably have thought of
that scenario already so you may have some contingency system in place or
maybe it's not a actual issue in your deployment situation? It woud be
great if you could elaborate a bit as it is always nice when people
exchange their best security practices.

One idea I had to prevent user to bypass Polaris authorization system would
be for Polaris itself to issue its own JWT tokens, and when accessing
storage, Polaris would do an internal exchange where claims from the
inbound token (received by Polaris) would be copied over to the outbound
token (sent to STS), after proper validation of course. Is it a scenario
you have considered by any chance?

Hope to hear more,

Cheers,

Laurent

On Thu, Nov 20, 2025 at 11:32 AM Arsenault, Reginald P. via dev <
dev@polaris.apache.org> wrote:

> UNCLASSIFIED / NON CLASSIFI?
>
> Hi! As per our findings in [2], we are looking to enable connectivity with
> our on-prem S3 storage appliance, which requires changes to how Polaris
> connects to S3. Currently, the way Polaris connects to S3 does not work for
> our on-prem S3 instance because of the way it expects to connect to the STS
> server (i.e. without any specific authentication).
> We propose enhancing the AwsCredentialsStorageIntegration to support a
> more flexible authorization mechanism:
>
>   *   Goal: We want the ability to pass the requesting user's identity
> (likely a token) to the STS. This would allow the STS to make fine-grained
> decisions on what the user can and cannot do based on their specific
> identity.
>   *   Proposed Implementation: We believe this functionality can be
> achieved by adding an option within AwsCredentialsStorageIntegration [1] to
> connect using the AssumeRoleWithWebIdentity flow.
> We would greatly appreciate some guidance on the following changes before
> we begin development:
>
>   1.  What is the recommended or existing mechanism within Polaris for
> capturing and passing along an authenticated user's identity token to the
> AwsCredentialsStorageIntegration component, so we can pass it to the STS? I
> could be wrong, but I believe the token that gets passed to STS doesn't
> need to be scoped for the STS, so it could just be whatever token is given
> to Polaris by the user.
>   2.  How should we best modify the AwsCredentialsStorageIntegration class
> and configuration to be able to handle both the existing AssumeRole use
> case and the new AssumeRoleWithWebIdentity use case?
> Any pointers or suggestions on the best place to inject the user token and
> manage the configuration switch would be helpful.
>
> [1]
> https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L107
> [2] https://github.com/apache/polaris/issues/3038
>