You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@polaris.apache.org by Alexandre Dutra <ad...@apache.org> on 2025/11/04 16:52:51 UTC

[DISCUSS] "Detached" principals

Hello everyone,

I'd like to propose an enhancement to Polaris's authentication
process: the introduction of "detached" (non-persisted) principals.

A detached principal is different from a federated principal. The
former isn't persisted at all, whereas the latter is persisted (with
federated flag = true), and possibly synchronized with some external
system (e.g. via SCIM).

Motivation:

Currently, Polaris requires principals to be pre-existing for
successful authentication – even those marked as "federated". The
default authenticator always performs a lookup by ID or name, and
fails if the principal isn't found.

This presents a challenge for users who utilize external
authentication and authorization systems, such as Keycloak + OPA,
where Polaris acts as a passthrough between authn and authz.

In such scenarios, principals still need to be created in advance
within Polaris, which doesn't make much sense.

Proposal:

1) Introduce a new property: PolarisCredential.isExternal().

2) During authentication, set isExternal=false on the
PolarisCredential if Polaris is the IDP (internal auth); otherwise,
set isExternal=true.

3) In DefaultAuthenticator:

    a) If PolarisCredential.isExternal() is false, continue with the
existing logic (lookups by ID and name).

    b) If PolarisCredential.isExternal() is true, perform the usual
lookups. If they succeed, continue with the existing logic. If they
fail though, don't fail the authentication. Instead, create the
principal and assign roles directly from the token.

4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
the principal and its roles (as this would fail for detached
principals).

I believe this change will not impact internal authentication with the
default authorizer. It also won't impact people using external IDPs
with pre-synchronized federated principals: the default authorizer
would work just fine.

However, for external IDPs combined with custom authorizers, this
change would significantly improve the user experience by eliminating
the need to persist the principal and its roles within Polaris.

What do you all think?

Thanks,
Alex

Re: [DISCUSS] "Detached" principals

Posted by Sung Yun <su...@apache.org>.
Hi all - thank you all for your review and interest on the proposal.

We discussed the proposal at today's Polaris Community Sync, and here's a summary of some notable points that were raised.

- There are concerns about introducing flags such as isExternal on PolarisPrincipal, as the principal model should not encode whether it was persisted in the metastore.
- It was also noted that the earlier “federated principal” concept emerged mainly because authorization previously depended on persisted PrincipalRoles, and that newer authorization plugins (like OpaPolarisAuthorizer) may support external identity without requiring persistence. So there is less necessity to introduce FederatedPrincipals, and limiting the number of variant code paths should be a consideration in our design.
- Another point raised was that the current Resolver always assumes metastore-backed principals, and that this behavior might be better expressed through a configurable abstraction rather than conditionals.

I will update the RFC to make note of these points, and explore an approach that decouples the metastore based principal resolution logic out of the Resolver and circulate the revised design for feedback on the mailing list.

Cheers,
Sung

On 2025/12/10 22:27:36 Yufei Gu wrote:
> Thanks for the design doc and PR, Sung! The doc looks good to me overall.
> Left some minor comments.
> 
> Yufei
> 
> 
> On Wed, Dec 10, 2025 at 1:57 PM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> 
> > Hi Sung,
> >
> > Thanks for writing down this proposal. I believe it fits nicely into the
> > previously discussed ideas of "detached" principals.
> >
> > I added some nitpicky / nomenclature comments in the doc, but overall it
> > LGTM.
> >
> > Cheers,
> > Dmitri.
> >
> > On Wed, Dec 10, 2025 at 4:20 PM Sung Yun <su...@apache.org> wrote:
> >
> > > Hi Alex, thank you again for driving this proposal.
> > >
> > > We have a real use case for "Detached" principals, so I took the liberty
> > > to write up this RFC[1] with a proposal to introduce an "External"
> > > Principal type, with a draft implementation plan based on your idea.
> > >
> > > Here's the draft PR based on this RFC[2]
> > >
> > > I'd appreciate a review from anyone who's interested in this topic!
> > >
> > > Cheers,
> > > Sung
> > >
> > > [1]
> > >
> > https://docs.google.com/document/d/1VSoN1-QsAJGaM40oWTxeLIoYTp-XlfQM9nG-jqfb_-E/edit?usp=sharing
> > > [2] https://github.com/apache/polaris/pull/3250
> > >
> > > On 2025/11/13 13:00:12 Alexandre Dutra wrote:
> > > > Hi all,
> > > >
> > > > Thanks for the excellent discussion so far! I added this topic to the
> > > > agenda for today's meeting, in order to get some alignment before
> > > > diving into a design doc or formal proposal. I hope that's OK for
> > > > everybody.
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <fl...@gmail.com> wrote:
> > > > >
> > > > > Thanks for the great discussion. +1 on clarifying the boundary and
> > > mutually
> > > > > exclusive modes, so that there is no confusion and chaos.
> > > > >
> > > > > Can we have a short design doc to facilitate further discussion?
> > > > > Particularly. it'd be nice to clarify authN and authZ modes Polaris
> > > > > supports like the following table:
> > > > >
> > > > > Mode Principal/Role persistence Privilege enforcement Who decides?
> > > > > *Native* Persisted Polaris RBAC Polaris
> > > > > *Federated* Persisted/synced Polaris RBAC Polaris
> > > > > *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External
> > > > >
> > > > > Once we clarified different modes, we could start to figure out the
> > > > > implications and correspondent changes. Things we might consider:
> > > > > 1. Do we allow mixed mode, e.g., some operations are with detached
> > > mode,
> > > > > others are with native mode? I guess this is pretty complex, we may
> > > not do
> > > > > that if there are no strong use cases.
> > > > > 2. How do we handle the audit in case of detached mode? Can we still
> > > get
> > > > > the consistent principal names?
> > > > >
> > > > > I'm happy to jump on a dedicated call for this.
> > > > >
> > > > > Yufei
> > > > >
> > > > >
> > > > > On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:
> > > > >
> > > > > > Thank you for suggesting this Alex, this is a great discussion.
> > > > > >
> > > > > > I see the value of persisted federated roles when Polaris is the
> > > primary
> > > > > > place where catalog privileges are managed and external IdPs just
> > > supply
> > > > > > identity and groups. In that model, stable role entities in Polaris
> > > are
> > > > > > useful and provide a clear governance boundary.
> > > > > >
> > > > > > For the OPA-backed PolarisAuthorizer use case, though, we are
> > moving
> > > > > > toward a setup where:
> > > > > > - Principals are not persisted in Polaris.
> > > > > > - Role and attribute evaluation happen in an external Policy
> > Decision
> > > > > > Point such as OPA.
> > > > > > - Polaris simply enforces authorization decisions on its REST
> > > resources.
> > > > > >
> > > > > > In this pattern, persisted federated roles become less compelling
> > > and can
> > > > > > even overlap with external policy configuration. So this “detached
> > > > > > principal” concept resonates with our use case… it makes the
> > > externalized
> > > > > > responsibility model explicit instead of overloading federated
> > roles
> > > to
> > > > > > serve both the Polaris-managed RBAC and externally-managed Policy
> > > Decision
> > > > > > Point use cases.
> > > > > >
> > > > > > So IMHO, it might be worth clarifying the model boundaries so both
> > > modes
> > > > > > are first-class:
> > > > > > - “Federated” for external identity with locally managed privileges
> > > in
> > > > > > Polaris.
> > > > > > - “Detached” for external identity with externally managed
> > > authorization
> > > > > > and no local persistence.
> > > > > >
> > > > > > That distinction could help keep the semantics clean while still
> > > > > > minimizing conceptual sprawl.
> > > > > >
> > > > > > If it would be helpful, would it make sense to organize a call to
> > > discuss
> > > > > > these concepts for those that are interested? (The usual Polaris
> > > Sync time
> > > > > > slot doesn’t work for me unfortunately, and I’d love to make a case
> > > for
> > > > > > this principal type to be supported)
> > > > > >
> > > > > > Cheers,
> > > > > > Sung
> > > > > >
> > > > > > On 2025/11/08 01:09:34 Michael Collado wrote:
> > > > > > > I see the distinction you're raising. It's a good point, though
> > I'm
> > > > > > > hesitant to introduce a third principal type to the code base.
> > The
> > > most
> > > > > > > notable thing about federated principals and roles is that role
> > > grants
> > > > > > are
> > > > > > > never managed through the Polaris API. This would be true even
> > for
> > > > > > > "detached" principals/roles. Federated principals (in the current
> > > code
> > > > > > > base) also can't be persisted (except possibly through some
> > > alternative
> > > > > > > process, such as SCIM). The main difference is that federated
> > > roles are
> > > > > > > expected to be persisted, but "detached" roles are not. Can we
> > > unify
> > > > > > these
> > > > > > > concepts so that we can support both ephemeral roles and
> > persisted
> > > roles
> > > > > > > without introducing a third type of role?
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <
> > adutra@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > > The code change in #1353 prohibited federated principals from
> > > being
> > > > > > > > persisted at all.
> > > > > > > >
> > > > > > > > Indeed! But I think it's important to distinguish between
> > > "federated"
> > > > > > > > and "detached". Here's how I see the difference:
> > > > > > > >
> > > > > > > > 1) Federated Principals and Roles
> > > > > > > >
> > > > > > > > This approach involves pre-synchronizing principals and roles
> > > with an
> > > > > > > > external system before they can access Polaris. This concept
> > was
> > > > > > > > outlined in an older design document [1] but never fully
> > > implemented.
> > > > > > > > Typically, this would be achieved using SCIM (eventually) or a
> > > > > > > > periodic synchronization script run by an administrator.
> > > > > > > >
> > > > > > > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > > > > > > just-in-time (JIT) creation of federated principals, meaning
> > the
> > > > > > > > principal would be materialized only upon first access. That's
> > > not the
> > > > > > > > case today though. I personally wonder if it would be simpler
> > to
> > > also
> > > > > > > > allow for principal synchronization via the management API.
> > (I'm
> > > happy
> > > > > > > > to resume this conversation later, but that's not the topic
> > > here.)
> > > > > > > >
> > > > > > > > 2) Detached Principals and Roles
> > > > > > > >
> > > > > > > > This scenario implies that no identity management information
> > > > > > > > whatsoever is stored in Polaris. This fits use cases where the
> > > > > > > > authorizer does not rely on Polaris's default privilege model:
> > > the OPA
> > > > > > > > authorizer is one example of that. This is the specific use
> > case
> > > > > > > > currently under discussion.
> > > > > > > >
> > > > > > > > Both flavors of principals are important for Polaris, and I
> > would
> > > > > > > > gladly resume the work on federated ones. But, it seems to me
> > > that
> > > > > > > > enabling detached principals is even more urgent today. WDYT?
> > > > > > > >
> > > > > > > > > I'd like this lookup to be covered by a feature flag [...] it
> > > would
> > > > > > be
> > > > > > > > nice to allow skipping the unnecessary database hit. WDYT?
> > > > > > > >
> > > > > > > > Yes, that is a very good suggestion!
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > [1]:
> > > > > > > >
> > > > > >
> > >
> > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > > > > > > [2]: https://github.com/apache/polaris/pull/1353
> > > > > > > >
> > > > > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> > > > > > collado.mike@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > The code change in #1353 prohibited federated principals from
> > > being
> > > > > > > > > persisted at all. Federated Principal Roles are persisted,
> > but
> > > at
> > > > > > this
> > > > > > > > time
> > > > > > > > > (AFAIK), we can't persist federated principals.
> > > > > > > > >
> > > > > > > > > Mike
> > > > > > > > >
> > > > > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <
> > > dimas@apache.org
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Alex,
> > > > > > > > > >
> > > > > > > > > > This proposal sounds very reasonable to me.
> > > > > > > > > >
> > > > > > > > > > I believe we had some recent user interest (in slack) for
> > > running
> > > > > > > > Polaris
> > > > > > > > > > with Keycloak. From my POV, implementing your proposal
> > should
> > > > > > > > > > significantly simplify that user's deployment.
> > > > > > > > > >
> > > > > > > > > > As for item (3b), I'd like this lookup to be covered by a
> > > feature
> > > > > > flag.
> > > > > > > > > > Admin users who do not import / sync Principals know that
> > > this
> > > > > > lookup
> > > > > > > > will
> > > > > > > > > > always fail, so it would be nice to allow skipping the
> > > unnecessary
> > > > > > > > database
> > > > > > > > > > hit. WDYT?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Dmitri.
> > > > > > > > > >
> > > > > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <
> > > adutra@apache.org
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello everyone,
> > > > > > > > > > >
> > > > > > > > > > > I'd like to propose an enhancement to Polaris's
> > > authentication
> > > > > > > > > > > process: the introduction of "detached" (non-persisted)
> > > > > > principals.
> > > > > > > > > > >
> > > > > > > > > > > A detached principal is different from a federated
> > > principal. The
> > > > > > > > > > > former isn't persisted at all, whereas the latter is
> > > persisted
> > > > > > (with
> > > > > > > > > > > federated flag = true), and possibly synchronized with
> > some
> > > > > > external
> > > > > > > > > > > system (e.g. via SCIM).
> > > > > > > > > > >
> > > > > > > > > > > Motivation:
> > > > > > > > > > >
> > > > > > > > > > > Currently, Polaris requires principals to be pre-existing
> > > for
> > > > > > > > > > > successful authentication – even those marked as
> > > "federated". The
> > > > > > > > > > > default authenticator always performs a lookup by ID or
> > > name, and
> > > > > > > > > > > fails if the principal isn't found.
> > > > > > > > > > >
> > > > > > > > > > > This presents a challenge for users who utilize external
> > > > > > > > > > > authentication and authorization systems, such as
> > Keycloak
> > > + OPA,
> > > > > > > > > > > where Polaris acts as a passthrough between authn and
> > > authz.
> > > > > > > > > > >
> > > > > > > > > > > In such scenarios, principals still need to be created in
> > > advance
> > > > > > > > > > > within Polaris, which doesn't make much sense.
> > > > > > > > > > >
> > > > > > > > > > > Proposal:
> > > > > > > > > > >
> > > > > > > > > > > 1) Introduce a new property:
> > > PolarisCredential.isExternal().
> > > > > > > > > > >
> > > > > > > > > > > 2) During authentication, set isExternal=false on the
> > > > > > > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> > > > > > otherwise,
> > > > > > > > > > > set isExternal=true.
> > > > > > > > > > >
> > > > > > > > > > > 3) In DefaultAuthenticator:
> > > > > > > > > > >
> > > > > > > > > > >     a) If PolarisCredential.isExternal() is false,
> > > continue with
> > > > > > the
> > > > > > > > > > > existing logic (lookups by ID and name).
> > > > > > > > > > >
> > > > > > > > > > >     b) If PolarisCredential.isExternal() is true, perform
> > > the
> > > > > > usual
> > > > > > > > > > > lookups. If they succeed, continue with the existing
> > > logic. If
> > > > > > they
> > > > > > > > > > > fail though, don't fail the authentication. Instead,
> > > create the
> > > > > > > > > > > principal and assign roles directly from the token.
> > > > > > > > > > >
> > > > > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> > > > > > re-resolving
> > > > > > > > > > > the principal and its roles (as this would fail for
> > > detached
> > > > > > > > > > > principals).
> > > > > > > > > > >
> > > > > > > > > > > I believe this change will not impact internal
> > > authentication
> > > > > > with
> > > > > > > > the
> > > > > > > > > > > default authorizer. It also won't impact people using
> > > external
> > > > > > IDPs
> > > > > > > > > > > with pre-synchronized federated principals: the default
> > > > > > authorizer
> > > > > > > > > > > would work just fine.
> > > > > > > > > > >
> > > > > > > > > > > However, for external IDPs combined with custom
> > > authorizers, this
> > > > > > > > > > > change would significantly improve the user experience by
> > > > > > eliminating
> > > > > > > > > > > the need to persist the principal and its roles within
> > > Polaris.
> > > > > > > > > > >
> > > > > > > > > > > What do you all think?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Alex
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> 

Re: [DISCUSS] "Detached" principals

Posted by Yufei Gu <fl...@gmail.com>.
Thanks for the design doc and PR, Sung! The doc looks good to me overall.
Left some minor comments.

Yufei


On Wed, Dec 10, 2025 at 1:57 PM Dmitri Bourlatchkov <di...@apache.org>
wrote:

> Hi Sung,
>
> Thanks for writing down this proposal. I believe it fits nicely into the
> previously discussed ideas of "detached" principals.
>
> I added some nitpicky / nomenclature comments in the doc, but overall it
> LGTM.
>
> Cheers,
> Dmitri.
>
> On Wed, Dec 10, 2025 at 4:20 PM Sung Yun <su...@apache.org> wrote:
>
> > Hi Alex, thank you again for driving this proposal.
> >
> > We have a real use case for "Detached" principals, so I took the liberty
> > to write up this RFC[1] with a proposal to introduce an "External"
> > Principal type, with a draft implementation plan based on your idea.
> >
> > Here's the draft PR based on this RFC[2]
> >
> > I'd appreciate a review from anyone who's interested in this topic!
> >
> > Cheers,
> > Sung
> >
> > [1]
> >
> https://docs.google.com/document/d/1VSoN1-QsAJGaM40oWTxeLIoYTp-XlfQM9nG-jqfb_-E/edit?usp=sharing
> > [2] https://github.com/apache/polaris/pull/3250
> >
> > On 2025/11/13 13:00:12 Alexandre Dutra wrote:
> > > Hi all,
> > >
> > > Thanks for the excellent discussion so far! I added this topic to the
> > > agenda for today's meeting, in order to get some alignment before
> > > diving into a design doc or formal proposal. I hope that's OK for
> > > everybody.
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <fl...@gmail.com> wrote:
> > > >
> > > > Thanks for the great discussion. +1 on clarifying the boundary and
> > mutually
> > > > exclusive modes, so that there is no confusion and chaos.
> > > >
> > > > Can we have a short design doc to facilitate further discussion?
> > > > Particularly. it'd be nice to clarify authN and authZ modes Polaris
> > > > supports like the following table:
> > > >
> > > > Mode Principal/Role persistence Privilege enforcement Who decides?
> > > > *Native* Persisted Polaris RBAC Polaris
> > > > *Federated* Persisted/synced Polaris RBAC Polaris
> > > > *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External
> > > >
> > > > Once we clarified different modes, we could start to figure out the
> > > > implications and correspondent changes. Things we might consider:
> > > > 1. Do we allow mixed mode, e.g., some operations are with detached
> > mode,
> > > > others are with native mode? I guess this is pretty complex, we may
> > not do
> > > > that if there are no strong use cases.
> > > > 2. How do we handle the audit in case of detached mode? Can we still
> > get
> > > > the consistent principal names?
> > > >
> > > > I'm happy to jump on a dedicated call for this.
> > > >
> > > > Yufei
> > > >
> > > >
> > > > On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:
> > > >
> > > > > Thank you for suggesting this Alex, this is a great discussion.
> > > > >
> > > > > I see the value of persisted federated roles when Polaris is the
> > primary
> > > > > place where catalog privileges are managed and external IdPs just
> > supply
> > > > > identity and groups. In that model, stable role entities in Polaris
> > are
> > > > > useful and provide a clear governance boundary.
> > > > >
> > > > > For the OPA-backed PolarisAuthorizer use case, though, we are
> moving
> > > > > toward a setup where:
> > > > > - Principals are not persisted in Polaris.
> > > > > - Role and attribute evaluation happen in an external Policy
> Decision
> > > > > Point such as OPA.
> > > > > - Polaris simply enforces authorization decisions on its REST
> > resources.
> > > > >
> > > > > In this pattern, persisted federated roles become less compelling
> > and can
> > > > > even overlap with external policy configuration. So this “detached
> > > > > principal” concept resonates with our use case… it makes the
> > externalized
> > > > > responsibility model explicit instead of overloading federated
> roles
> > to
> > > > > serve both the Polaris-managed RBAC and externally-managed Policy
> > Decision
> > > > > Point use cases.
> > > > >
> > > > > So IMHO, it might be worth clarifying the model boundaries so both
> > modes
> > > > > are first-class:
> > > > > - “Federated” for external identity with locally managed privileges
> > in
> > > > > Polaris.
> > > > > - “Detached” for external identity with externally managed
> > authorization
> > > > > and no local persistence.
> > > > >
> > > > > That distinction could help keep the semantics clean while still
> > > > > minimizing conceptual sprawl.
> > > > >
> > > > > If it would be helpful, would it make sense to organize a call to
> > discuss
> > > > > these concepts for those that are interested? (The usual Polaris
> > Sync time
> > > > > slot doesn’t work for me unfortunately, and I’d love to make a case
> > for
> > > > > this principal type to be supported)
> > > > >
> > > > > Cheers,
> > > > > Sung
> > > > >
> > > > > On 2025/11/08 01:09:34 Michael Collado wrote:
> > > > > > I see the distinction you're raising. It's a good point, though
> I'm
> > > > > > hesitant to introduce a third principal type to the code base.
> The
> > most
> > > > > > notable thing about federated principals and roles is that role
> > grants
> > > > > are
> > > > > > never managed through the Polaris API. This would be true even
> for
> > > > > > "detached" principals/roles. Federated principals (in the current
> > code
> > > > > > base) also can't be persisted (except possibly through some
> > alternative
> > > > > > process, such as SCIM). The main difference is that federated
> > roles are
> > > > > > expected to be persisted, but "detached" roles are not. Can we
> > unify
> > > > > these
> > > > > > concepts so that we can support both ephemeral roles and
> persisted
> > roles
> > > > > > without introducing a third type of role?
> > > > > >
> > > > > > Mike
> > > > > >
> > > > > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <
> adutra@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > > The code change in #1353 prohibited federated principals from
> > being
> > > > > > > persisted at all.
> > > > > > >
> > > > > > > Indeed! But I think it's important to distinguish between
> > "federated"
> > > > > > > and "detached". Here's how I see the difference:
> > > > > > >
> > > > > > > 1) Federated Principals and Roles
> > > > > > >
> > > > > > > This approach involves pre-synchronizing principals and roles
> > with an
> > > > > > > external system before they can access Polaris. This concept
> was
> > > > > > > outlined in an older design document [1] but never fully
> > implemented.
> > > > > > > Typically, this would be achieved using SCIM (eventually) or a
> > > > > > > periodic synchronization script run by an administrator.
> > > > > > >
> > > > > > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > > > > > just-in-time (JIT) creation of federated principals, meaning
> the
> > > > > > > principal would be materialized only upon first access. That's
> > not the
> > > > > > > case today though. I personally wonder if it would be simpler
> to
> > also
> > > > > > > allow for principal synchronization via the management API.
> (I'm
> > happy
> > > > > > > to resume this conversation later, but that's not the topic
> > here.)
> > > > > > >
> > > > > > > 2) Detached Principals and Roles
> > > > > > >
> > > > > > > This scenario implies that no identity management information
> > > > > > > whatsoever is stored in Polaris. This fits use cases where the
> > > > > > > authorizer does not rely on Polaris's default privilege model:
> > the OPA
> > > > > > > authorizer is one example of that. This is the specific use
> case
> > > > > > > currently under discussion.
> > > > > > >
> > > > > > > Both flavors of principals are important for Polaris, and I
> would
> > > > > > > gladly resume the work on federated ones. But, it seems to me
> > that
> > > > > > > enabling detached principals is even more urgent today. WDYT?
> > > > > > >
> > > > > > > > I'd like this lookup to be covered by a feature flag [...] it
> > would
> > > > > be
> > > > > > > nice to allow skipping the unnecessary database hit. WDYT?
> > > > > > >
> > > > > > > Yes, that is a very good suggestion!
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > > > [1]:
> > > > > > >
> > > > >
> >
> https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > > > > > [2]: https://github.com/apache/polaris/pull/1353
> > > > > > >
> > > > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> > > > > collado.mike@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > The code change in #1353 prohibited federated principals from
> > being
> > > > > > > > persisted at all. Federated Principal Roles are persisted,
> but
> > at
> > > > > this
> > > > > > > time
> > > > > > > > (AFAIK), we can't persist federated principals.
> > > > > > > >
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <
> > dimas@apache.org
> > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Alex,
> > > > > > > > >
> > > > > > > > > This proposal sounds very reasonable to me.
> > > > > > > > >
> > > > > > > > > I believe we had some recent user interest (in slack) for
> > running
> > > > > > > Polaris
> > > > > > > > > with Keycloak. From my POV, implementing your proposal
> should
> > > > > > > > > significantly simplify that user's deployment.
> > > > > > > > >
> > > > > > > > > As for item (3b), I'd like this lookup to be covered by a
> > feature
> > > > > flag.
> > > > > > > > > Admin users who do not import / sync Principals know that
> > this
> > > > > lookup
> > > > > > > will
> > > > > > > > > always fail, so it would be nice to allow skipping the
> > unnecessary
> > > > > > > database
> > > > > > > > > hit. WDYT?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Dmitri.
> > > > > > > > >
> > > > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <
> > adutra@apache.org
> > > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello everyone,
> > > > > > > > > >
> > > > > > > > > > I'd like to propose an enhancement to Polaris's
> > authentication
> > > > > > > > > > process: the introduction of "detached" (non-persisted)
> > > > > principals.
> > > > > > > > > >
> > > > > > > > > > A detached principal is different from a federated
> > principal. The
> > > > > > > > > > former isn't persisted at all, whereas the latter is
> > persisted
> > > > > (with
> > > > > > > > > > federated flag = true), and possibly synchronized with
> some
> > > > > external
> > > > > > > > > > system (e.g. via SCIM).
> > > > > > > > > >
> > > > > > > > > > Motivation:
> > > > > > > > > >
> > > > > > > > > > Currently, Polaris requires principals to be pre-existing
> > for
> > > > > > > > > > successful authentication – even those marked as
> > "federated". The
> > > > > > > > > > default authenticator always performs a lookup by ID or
> > name, and
> > > > > > > > > > fails if the principal isn't found.
> > > > > > > > > >
> > > > > > > > > > This presents a challenge for users who utilize external
> > > > > > > > > > authentication and authorization systems, such as
> Keycloak
> > + OPA,
> > > > > > > > > > where Polaris acts as a passthrough between authn and
> > authz.
> > > > > > > > > >
> > > > > > > > > > In such scenarios, principals still need to be created in
> > advance
> > > > > > > > > > within Polaris, which doesn't make much sense.
> > > > > > > > > >
> > > > > > > > > > Proposal:
> > > > > > > > > >
> > > > > > > > > > 1) Introduce a new property:
> > PolarisCredential.isExternal().
> > > > > > > > > >
> > > > > > > > > > 2) During authentication, set isExternal=false on the
> > > > > > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> > > > > otherwise,
> > > > > > > > > > set isExternal=true.
> > > > > > > > > >
> > > > > > > > > > 3) In DefaultAuthenticator:
> > > > > > > > > >
> > > > > > > > > >     a) If PolarisCredential.isExternal() is false,
> > continue with
> > > > > the
> > > > > > > > > > existing logic (lookups by ID and name).
> > > > > > > > > >
> > > > > > > > > >     b) If PolarisCredential.isExternal() is true, perform
> > the
> > > > > usual
> > > > > > > > > > lookups. If they succeed, continue with the existing
> > logic. If
> > > > > they
> > > > > > > > > > fail though, don't fail the authentication. Instead,
> > create the
> > > > > > > > > > principal and assign roles directly from the token.
> > > > > > > > > >
> > > > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> > > > > re-resolving
> > > > > > > > > > the principal and its roles (as this would fail for
> > detached
> > > > > > > > > > principals).
> > > > > > > > > >
> > > > > > > > > > I believe this change will not impact internal
> > authentication
> > > > > with
> > > > > > > the
> > > > > > > > > > default authorizer. It also won't impact people using
> > external
> > > > > IDPs
> > > > > > > > > > with pre-synchronized federated principals: the default
> > > > > authorizer
> > > > > > > > > > would work just fine.
> > > > > > > > > >
> > > > > > > > > > However, for external IDPs combined with custom
> > authorizers, this
> > > > > > > > > > change would significantly improve the user experience by
> > > > > eliminating
> > > > > > > > > > the need to persist the principal and its roles within
> > Polaris.
> > > > > > > > > >
> > > > > > > > > > What do you all think?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Alex
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>

Re: [DISCUSS] "Detached" principals

Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Sung,

Thanks for writing down this proposal. I believe it fits nicely into the
previously discussed ideas of "detached" principals.

I added some nitpicky / nomenclature comments in the doc, but overall it
LGTM.

Cheers,
Dmitri.

On Wed, Dec 10, 2025 at 4:20 PM Sung Yun <su...@apache.org> wrote:

> Hi Alex, thank you again for driving this proposal.
>
> We have a real use case for "Detached" principals, so I took the liberty
> to write up this RFC[1] with a proposal to introduce an "External"
> Principal type, with a draft implementation plan based on your idea.
>
> Here's the draft PR based on this RFC[2]
>
> I'd appreciate a review from anyone who's interested in this topic!
>
> Cheers,
> Sung
>
> [1]
> https://docs.google.com/document/d/1VSoN1-QsAJGaM40oWTxeLIoYTp-XlfQM9nG-jqfb_-E/edit?usp=sharing
> [2] https://github.com/apache/polaris/pull/3250
>
> On 2025/11/13 13:00:12 Alexandre Dutra wrote:
> > Hi all,
> >
> > Thanks for the excellent discussion so far! I added this topic to the
> > agenda for today's meeting, in order to get some alignment before
> > diving into a design doc or formal proposal. I hope that's OK for
> > everybody.
> >
> > Thanks,
> > Alex
> >
> > On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <fl...@gmail.com> wrote:
> > >
> > > Thanks for the great discussion. +1 on clarifying the boundary and
> mutually
> > > exclusive modes, so that there is no confusion and chaos.
> > >
> > > Can we have a short design doc to facilitate further discussion?
> > > Particularly. it'd be nice to clarify authN and authZ modes Polaris
> > > supports like the following table:
> > >
> > > Mode Principal/Role persistence Privilege enforcement Who decides?
> > > *Native* Persisted Polaris RBAC Polaris
> > > *Federated* Persisted/synced Polaris RBAC Polaris
> > > *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External
> > >
> > > Once we clarified different modes, we could start to figure out the
> > > implications and correspondent changes. Things we might consider:
> > > 1. Do we allow mixed mode, e.g., some operations are with detached
> mode,
> > > others are with native mode? I guess this is pretty complex, we may
> not do
> > > that if there are no strong use cases.
> > > 2. How do we handle the audit in case of detached mode? Can we still
> get
> > > the consistent principal names?
> > >
> > > I'm happy to jump on a dedicated call for this.
> > >
> > > Yufei
> > >
> > >
> > > On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:
> > >
> > > > Thank you for suggesting this Alex, this is a great discussion.
> > > >
> > > > I see the value of persisted federated roles when Polaris is the
> primary
> > > > place where catalog privileges are managed and external IdPs just
> supply
> > > > identity and groups. In that model, stable role entities in Polaris
> are
> > > > useful and provide a clear governance boundary.
> > > >
> > > > For the OPA-backed PolarisAuthorizer use case, though, we are moving
> > > > toward a setup where:
> > > > - Principals are not persisted in Polaris.
> > > > - Role and attribute evaluation happen in an external Policy Decision
> > > > Point such as OPA.
> > > > - Polaris simply enforces authorization decisions on its REST
> resources.
> > > >
> > > > In this pattern, persisted federated roles become less compelling
> and can
> > > > even overlap with external policy configuration. So this “detached
> > > > principal” concept resonates with our use case… it makes the
> externalized
> > > > responsibility model explicit instead of overloading federated roles
> to
> > > > serve both the Polaris-managed RBAC and externally-managed Policy
> Decision
> > > > Point use cases.
> > > >
> > > > So IMHO, it might be worth clarifying the model boundaries so both
> modes
> > > > are first-class:
> > > > - “Federated” for external identity with locally managed privileges
> in
> > > > Polaris.
> > > > - “Detached” for external identity with externally managed
> authorization
> > > > and no local persistence.
> > > >
> > > > That distinction could help keep the semantics clean while still
> > > > minimizing conceptual sprawl.
> > > >
> > > > If it would be helpful, would it make sense to organize a call to
> discuss
> > > > these concepts for those that are interested? (The usual Polaris
> Sync time
> > > > slot doesn’t work for me unfortunately, and I’d love to make a case
> for
> > > > this principal type to be supported)
> > > >
> > > > Cheers,
> > > > Sung
> > > >
> > > > On 2025/11/08 01:09:34 Michael Collado wrote:
> > > > > I see the distinction you're raising. It's a good point, though I'm
> > > > > hesitant to introduce a third principal type to the code base. The
> most
> > > > > notable thing about federated principals and roles is that role
> grants
> > > > are
> > > > > never managed through the Polaris API. This would be true even for
> > > > > "detached" principals/roles. Federated principals (in the current
> code
> > > > > base) also can't be persisted (except possibly through some
> alternative
> > > > > process, such as SCIM). The main difference is that federated
> roles are
> > > > > expected to be persisted, but "detached" roles are not. Can we
> unify
> > > > these
> > > > > concepts so that we can support both ephemeral roles and persisted
> roles
> > > > > without introducing a third type of role?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org>
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > > The code change in #1353 prohibited federated principals from
> being
> > > > > > persisted at all.
> > > > > >
> > > > > > Indeed! But I think it's important to distinguish between
> "federated"
> > > > > > and "detached". Here's how I see the difference:
> > > > > >
> > > > > > 1) Federated Principals and Roles
> > > > > >
> > > > > > This approach involves pre-synchronizing principals and roles
> with an
> > > > > > external system before they can access Polaris. This concept was
> > > > > > outlined in an older design document [1] but never fully
> implemented.
> > > > > > Typically, this would be achieved using SCIM (eventually) or a
> > > > > > periodic synchronization script run by an administrator.
> > > > > >
> > > > > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > > > > just-in-time (JIT) creation of federated principals, meaning the
> > > > > > principal would be materialized only upon first access. That's
> not the
> > > > > > case today though. I personally wonder if it would be simpler to
> also
> > > > > > allow for principal synchronization via the management API. (I'm
> happy
> > > > > > to resume this conversation later, but that's not the topic
> here.)
> > > > > >
> > > > > > 2) Detached Principals and Roles
> > > > > >
> > > > > > This scenario implies that no identity management information
> > > > > > whatsoever is stored in Polaris. This fits use cases where the
> > > > > > authorizer does not rely on Polaris's default privilege model:
> the OPA
> > > > > > authorizer is one example of that. This is the specific use case
> > > > > > currently under discussion.
> > > > > >
> > > > > > Both flavors of principals are important for Polaris, and I would
> > > > > > gladly resume the work on federated ones. But, it seems to me
> that
> > > > > > enabling detached principals is even more urgent today. WDYT?
> > > > > >
> > > > > > > I'd like this lookup to be covered by a feature flag [...] it
> would
> > > > be
> > > > > > nice to allow skipping the unnecessary database hit. WDYT?
> > > > > >
> > > > > > Yes, that is a very good suggestion!
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > > > [1]:
> > > > > >
> > > >
> https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > > > > [2]: https://github.com/apache/polaris/pull/1353
> > > > > >
> > > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> > > > collado.mike@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > The code change in #1353 prohibited federated principals from
> being
> > > > > > > persisted at all. Federated Principal Roles are persisted, but
> at
> > > > this
> > > > > > time
> > > > > > > (AFAIK), we can't persist federated principals.
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <
> dimas@apache.org
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Alex,
> > > > > > > >
> > > > > > > > This proposal sounds very reasonable to me.
> > > > > > > >
> > > > > > > > I believe we had some recent user interest (in slack) for
> running
> > > > > > Polaris
> > > > > > > > with Keycloak. From my POV, implementing your proposal should
> > > > > > > > significantly simplify that user's deployment.
> > > > > > > >
> > > > > > > > As for item (3b), I'd like this lookup to be covered by a
> feature
> > > > flag.
> > > > > > > > Admin users who do not import / sync Principals know that
> this
> > > > lookup
> > > > > > will
> > > > > > > > always fail, so it would be nice to allow skipping the
> unnecessary
> > > > > > database
> > > > > > > > hit. WDYT?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Dmitri.
> > > > > > > >
> > > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <
> adutra@apache.org
> > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello everyone,
> > > > > > > > >
> > > > > > > > > I'd like to propose an enhancement to Polaris's
> authentication
> > > > > > > > > process: the introduction of "detached" (non-persisted)
> > > > principals.
> > > > > > > > >
> > > > > > > > > A detached principal is different from a federated
> principal. The
> > > > > > > > > former isn't persisted at all, whereas the latter is
> persisted
> > > > (with
> > > > > > > > > federated flag = true), and possibly synchronized with some
> > > > external
> > > > > > > > > system (e.g. via SCIM).
> > > > > > > > >
> > > > > > > > > Motivation:
> > > > > > > > >
> > > > > > > > > Currently, Polaris requires principals to be pre-existing
> for
> > > > > > > > > successful authentication – even those marked as
> "federated". The
> > > > > > > > > default authenticator always performs a lookup by ID or
> name, and
> > > > > > > > > fails if the principal isn't found.
> > > > > > > > >
> > > > > > > > > This presents a challenge for users who utilize external
> > > > > > > > > authentication and authorization systems, such as Keycloak
> + OPA,
> > > > > > > > > where Polaris acts as a passthrough between authn and
> authz.
> > > > > > > > >
> > > > > > > > > In such scenarios, principals still need to be created in
> advance
> > > > > > > > > within Polaris, which doesn't make much sense.
> > > > > > > > >
> > > > > > > > > Proposal:
> > > > > > > > >
> > > > > > > > > 1) Introduce a new property:
> PolarisCredential.isExternal().
> > > > > > > > >
> > > > > > > > > 2) During authentication, set isExternal=false on the
> > > > > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> > > > otherwise,
> > > > > > > > > set isExternal=true.
> > > > > > > > >
> > > > > > > > > 3) In DefaultAuthenticator:
> > > > > > > > >
> > > > > > > > >     a) If PolarisCredential.isExternal() is false,
> continue with
> > > > the
> > > > > > > > > existing logic (lookups by ID and name).
> > > > > > > > >
> > > > > > > > >     b) If PolarisCredential.isExternal() is true, perform
> the
> > > > usual
> > > > > > > > > lookups. If they succeed, continue with the existing
> logic. If
> > > > they
> > > > > > > > > fail though, don't fail the authentication. Instead,
> create the
> > > > > > > > > principal and assign roles directly from the token.
> > > > > > > > >
> > > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> > > > re-resolving
> > > > > > > > > the principal and its roles (as this would fail for
> detached
> > > > > > > > > principals).
> > > > > > > > >
> > > > > > > > > I believe this change will not impact internal
> authentication
> > > > with
> > > > > > the
> > > > > > > > > default authorizer. It also won't impact people using
> external
> > > > IDPs
> > > > > > > > > with pre-synchronized federated principals: the default
> > > > authorizer
> > > > > > > > > would work just fine.
> > > > > > > > >
> > > > > > > > > However, for external IDPs combined with custom
> authorizers, this
> > > > > > > > > change would significantly improve the user experience by
> > > > eliminating
> > > > > > > > > the need to persist the principal and its roles within
> Polaris.
> > > > > > > > >
> > > > > > > > > What do you all think?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> >
>

Re: [DISCUSS] "Detached" principals

Posted by Sung Yun <su...@apache.org>.
Hi Alex, thank you again for driving this proposal.

We have a real use case for "Detached" principals, so I took the liberty to write up this RFC[1] with a proposal to introduce an "External" Principal type, with a draft implementation plan based on your idea.

Here's the draft PR based on this RFC[2]

I'd appreciate a review from anyone who's interested in this topic!

Cheers,
Sung

[1] https://docs.google.com/document/d/1VSoN1-QsAJGaM40oWTxeLIoYTp-XlfQM9nG-jqfb_-E/edit?usp=sharing
[2] https://github.com/apache/polaris/pull/3250

On 2025/11/13 13:00:12 Alexandre Dutra wrote:
> Hi all,
> 
> Thanks for the excellent discussion so far! I added this topic to the
> agenda for today's meeting, in order to get some alignment before
> diving into a design doc or formal proposal. I hope that's OK for
> everybody.
> 
> Thanks,
> Alex
> 
> On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <fl...@gmail.com> wrote:
> >
> > Thanks for the great discussion. +1 on clarifying the boundary and mutually
> > exclusive modes, so that there is no confusion and chaos.
> >
> > Can we have a short design doc to facilitate further discussion?
> > Particularly. it'd be nice to clarify authN and authZ modes Polaris
> > supports like the following table:
> >
> > Mode Principal/Role persistence Privilege enforcement Who decides?
> > *Native* Persisted Polaris RBAC Polaris
> > *Federated* Persisted/synced Polaris RBAC Polaris
> > *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External
> >
> > Once we clarified different modes, we could start to figure out the
> > implications and correspondent changes. Things we might consider:
> > 1. Do we allow mixed mode, e.g., some operations are with detached mode,
> > others are with native mode? I guess this is pretty complex, we may not do
> > that if there are no strong use cases.
> > 2. How do we handle the audit in case of detached mode? Can we still get
> > the consistent principal names?
> >
> > I'm happy to jump on a dedicated call for this.
> >
> > Yufei
> >
> >
> > On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:
> >
> > > Thank you for suggesting this Alex, this is a great discussion.
> > >
> > > I see the value of persisted federated roles when Polaris is the primary
> > > place where catalog privileges are managed and external IdPs just supply
> > > identity and groups. In that model, stable role entities in Polaris are
> > > useful and provide a clear governance boundary.
> > >
> > > For the OPA-backed PolarisAuthorizer use case, though, we are moving
> > > toward a setup where:
> > > - Principals are not persisted in Polaris.
> > > - Role and attribute evaluation happen in an external Policy Decision
> > > Point such as OPA.
> > > - Polaris simply enforces authorization decisions on its REST resources.
> > >
> > > In this pattern, persisted federated roles become less compelling and can
> > > even overlap with external policy configuration. So this “detached
> > > principal” concept resonates with our use case… it makes the externalized
> > > responsibility model explicit instead of overloading federated roles to
> > > serve both the Polaris-managed RBAC and externally-managed Policy Decision
> > > Point use cases.
> > >
> > > So IMHO, it might be worth clarifying the model boundaries so both modes
> > > are first-class:
> > > - “Federated” for external identity with locally managed privileges in
> > > Polaris.
> > > - “Detached” for external identity with externally managed authorization
> > > and no local persistence.
> > >
> > > That distinction could help keep the semantics clean while still
> > > minimizing conceptual sprawl.
> > >
> > > If it would be helpful, would it make sense to organize a call to discuss
> > > these concepts for those that are interested? (The usual Polaris Sync time
> > > slot doesn’t work for me unfortunately, and I’d love to make a case for
> > > this principal type to be supported)
> > >
> > > Cheers,
> > > Sung
> > >
> > > On 2025/11/08 01:09:34 Michael Collado wrote:
> > > > I see the distinction you're raising. It's a good point, though I'm
> > > > hesitant to introduce a third principal type to the code base. The most
> > > > notable thing about federated principals and roles is that role grants
> > > are
> > > > never managed through the Polaris API. This would be true even for
> > > > "detached" principals/roles. Federated principals (in the current code
> > > > base) also can't be persisted (except possibly through some alternative
> > > > process, such as SCIM). The main difference is that federated roles are
> > > > expected to be persisted, but "detached" roles are not. Can we unify
> > > these
> > > > concepts so that we can support both ephemeral roles and persisted roles
> > > > without introducing a third type of role?
> > > >
> > > > Mike
> > > >
> > > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > > The code change in #1353 prohibited federated principals from being
> > > > > persisted at all.
> > > > >
> > > > > Indeed! But I think it's important to distinguish between "federated"
> > > > > and "detached". Here's how I see the difference:
> > > > >
> > > > > 1) Federated Principals and Roles
> > > > >
> > > > > This approach involves pre-synchronizing principals and roles with an
> > > > > external system before they can access Polaris. This concept was
> > > > > outlined in an older design document [1] but never fully implemented.
> > > > > Typically, this would be achieved using SCIM (eventually) or a
> > > > > periodic synchronization script run by an administrator.
> > > > >
> > > > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > > > just-in-time (JIT) creation of federated principals, meaning the
> > > > > principal would be materialized only upon first access. That's not the
> > > > > case today though. I personally wonder if it would be simpler to also
> > > > > allow for principal synchronization via the management API. (I'm happy
> > > > > to resume this conversation later, but that's not the topic here.)
> > > > >
> > > > > 2) Detached Principals and Roles
> > > > >
> > > > > This scenario implies that no identity management information
> > > > > whatsoever is stored in Polaris. This fits use cases where the
> > > > > authorizer does not rely on Polaris's default privilege model: the OPA
> > > > > authorizer is one example of that. This is the specific use case
> > > > > currently under discussion.
> > > > >
> > > > > Both flavors of principals are important for Polaris, and I would
> > > > > gladly resume the work on federated ones. But, it seems to me that
> > > > > enabling detached principals is even more urgent today. WDYT?
> > > > >
> > > > > > I'd like this lookup to be covered by a feature flag [...] it would
> > > be
> > > > > nice to allow skipping the unnecessary database hit. WDYT?
> > > > >
> > > > > Yes, that is a very good suggestion!
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > > [1]:
> > > > >
> > > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > > > [2]: https://github.com/apache/polaris/pull/1353
> > > > >
> > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> > > collado.mike@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > The code change in #1353 prohibited federated principals from being
> > > > > > persisted at all. Federated Principal Roles are persisted, but at
> > > this
> > > > > time
> > > > > > (AFAIK), we can't persist federated principals.
> > > > > >
> > > > > > Mike
> > > > > >
> > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <dimas@apache.org
> > > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi Alex,
> > > > > > >
> > > > > > > This proposal sounds very reasonable to me.
> > > > > > >
> > > > > > > I believe we had some recent user interest (in slack) for running
> > > > > Polaris
> > > > > > > with Keycloak. From my POV, implementing your proposal should
> > > > > > > significantly simplify that user's deployment.
> > > > > > >
> > > > > > > As for item (3b), I'd like this lookup to be covered by a feature
> > > flag.
> > > > > > > Admin users who do not import / sync Principals know that this
> > > lookup
> > > > > will
> > > > > > > always fail, so it would be nice to allow skipping the unnecessary
> > > > > database
> > > > > > > hit. WDYT?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Dmitri.
> > > > > > >
> > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <adutra@apache.org
> > > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hello everyone,
> > > > > > > >
> > > > > > > > I'd like to propose an enhancement to Polaris's authentication
> > > > > > > > process: the introduction of "detached" (non-persisted)
> > > principals.
> > > > > > > >
> > > > > > > > A detached principal is different from a federated principal. The
> > > > > > > > former isn't persisted at all, whereas the latter is persisted
> > > (with
> > > > > > > > federated flag = true), and possibly synchronized with some
> > > external
> > > > > > > > system (e.g. via SCIM).
> > > > > > > >
> > > > > > > > Motivation:
> > > > > > > >
> > > > > > > > Currently, Polaris requires principals to be pre-existing for
> > > > > > > > successful authentication – even those marked as "federated". The
> > > > > > > > default authenticator always performs a lookup by ID or name, and
> > > > > > > > fails if the principal isn't found.
> > > > > > > >
> > > > > > > > This presents a challenge for users who utilize external
> > > > > > > > authentication and authorization systems, such as Keycloak + OPA,
> > > > > > > > where Polaris acts as a passthrough between authn and authz.
> > > > > > > >
> > > > > > > > In such scenarios, principals still need to be created in advance
> > > > > > > > within Polaris, which doesn't make much sense.
> > > > > > > >
> > > > > > > > Proposal:
> > > > > > > >
> > > > > > > > 1) Introduce a new property: PolarisCredential.isExternal().
> > > > > > > >
> > > > > > > > 2) During authentication, set isExternal=false on the
> > > > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> > > otherwise,
> > > > > > > > set isExternal=true.
> > > > > > > >
> > > > > > > > 3) In DefaultAuthenticator:
> > > > > > > >
> > > > > > > >     a) If PolarisCredential.isExternal() is false, continue with
> > > the
> > > > > > > > existing logic (lookups by ID and name).
> > > > > > > >
> > > > > > > >     b) If PolarisCredential.isExternal() is true, perform the
> > > usual
> > > > > > > > lookups. If they succeed, continue with the existing logic. If
> > > they
> > > > > > > > fail though, don't fail the authentication. Instead, create the
> > > > > > > > principal and assign roles directly from the token.
> > > > > > > >
> > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> > > re-resolving
> > > > > > > > the principal and its roles (as this would fail for detached
> > > > > > > > principals).
> > > > > > > >
> > > > > > > > I believe this change will not impact internal authentication
> > > with
> > > > > the
> > > > > > > > default authorizer. It also won't impact people using external
> > > IDPs
> > > > > > > > with pre-synchronized federated principals: the default
> > > authorizer
> > > > > > > > would work just fine.
> > > > > > > >
> > > > > > > > However, for external IDPs combined with custom authorizers, this
> > > > > > > > change would significantly improve the user experience by
> > > eliminating
> > > > > > > > the need to persist the principal and its roles within Polaris.
> > > > > > > >
> > > > > > > > What do you all think?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alex
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> 

Re: [DISCUSS] "Detached" principals

Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,

Thanks for the excellent discussion so far! I added this topic to the
agenda for today's meeting, in order to get some alignment before
diving into a design doc or formal proposal. I hope that's OK for
everybody.

Thanks,
Alex

On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <fl...@gmail.com> wrote:
>
> Thanks for the great discussion. +1 on clarifying the boundary and mutually
> exclusive modes, so that there is no confusion and chaos.
>
> Can we have a short design doc to facilitate further discussion?
> Particularly. it'd be nice to clarify authN and authZ modes Polaris
> supports like the following table:
>
> Mode Principal/Role persistence Privilege enforcement Who decides?
> *Native* Persisted Polaris RBAC Polaris
> *Federated* Persisted/synced Polaris RBAC Polaris
> *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External
>
> Once we clarified different modes, we could start to figure out the
> implications and correspondent changes. Things we might consider:
> 1. Do we allow mixed mode, e.g., some operations are with detached mode,
> others are with native mode? I guess this is pretty complex, we may not do
> that if there are no strong use cases.
> 2. How do we handle the audit in case of detached mode? Can we still get
> the consistent principal names?
>
> I'm happy to jump on a dedicated call for this.
>
> Yufei
>
>
> On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:
>
> > Thank you for suggesting this Alex, this is a great discussion.
> >
> > I see the value of persisted federated roles when Polaris is the primary
> > place where catalog privileges are managed and external IdPs just supply
> > identity and groups. In that model, stable role entities in Polaris are
> > useful and provide a clear governance boundary.
> >
> > For the OPA-backed PolarisAuthorizer use case, though, we are moving
> > toward a setup where:
> > - Principals are not persisted in Polaris.
> > - Role and attribute evaluation happen in an external Policy Decision
> > Point such as OPA.
> > - Polaris simply enforces authorization decisions on its REST resources.
> >
> > In this pattern, persisted federated roles become less compelling and can
> > even overlap with external policy configuration. So this “detached
> > principal” concept resonates with our use case… it makes the externalized
> > responsibility model explicit instead of overloading federated roles to
> > serve both the Polaris-managed RBAC and externally-managed Policy Decision
> > Point use cases.
> >
> > So IMHO, it might be worth clarifying the model boundaries so both modes
> > are first-class:
> > - “Federated” for external identity with locally managed privileges in
> > Polaris.
> > - “Detached” for external identity with externally managed authorization
> > and no local persistence.
> >
> > That distinction could help keep the semantics clean while still
> > minimizing conceptual sprawl.
> >
> > If it would be helpful, would it make sense to organize a call to discuss
> > these concepts for those that are interested? (The usual Polaris Sync time
> > slot doesn’t work for me unfortunately, and I’d love to make a case for
> > this principal type to be supported)
> >
> > Cheers,
> > Sung
> >
> > On 2025/11/08 01:09:34 Michael Collado wrote:
> > > I see the distinction you're raising. It's a good point, though I'm
> > > hesitant to introduce a third principal type to the code base. The most
> > > notable thing about federated principals and roles is that role grants
> > are
> > > never managed through the Polaris API. This would be true even for
> > > "detached" principals/roles. Federated principals (in the current code
> > > base) also can't be persisted (except possibly through some alternative
> > > process, such as SCIM). The main difference is that federated roles are
> > > expected to be persisted, but "detached" roles are not. Can we unify
> > these
> > > concepts so that we can support both ephemeral roles and persisted roles
> > > without introducing a third type of role?
> > >
> > > Mike
> > >
> > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > > The code change in #1353 prohibited federated principals from being
> > > > persisted at all.
> > > >
> > > > Indeed! But I think it's important to distinguish between "federated"
> > > > and "detached". Here's how I see the difference:
> > > >
> > > > 1) Federated Principals and Roles
> > > >
> > > > This approach involves pre-synchronizing principals and roles with an
> > > > external system before they can access Polaris. This concept was
> > > > outlined in an older design document [1] but never fully implemented.
> > > > Typically, this would be achieved using SCIM (eventually) or a
> > > > periodic synchronization script run by an administrator.
> > > >
> > > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > > just-in-time (JIT) creation of federated principals, meaning the
> > > > principal would be materialized only upon first access. That's not the
> > > > case today though. I personally wonder if it would be simpler to also
> > > > allow for principal synchronization via the management API. (I'm happy
> > > > to resume this conversation later, but that's not the topic here.)
> > > >
> > > > 2) Detached Principals and Roles
> > > >
> > > > This scenario implies that no identity management information
> > > > whatsoever is stored in Polaris. This fits use cases where the
> > > > authorizer does not rely on Polaris's default privilege model: the OPA
> > > > authorizer is one example of that. This is the specific use case
> > > > currently under discussion.
> > > >
> > > > Both flavors of principals are important for Polaris, and I would
> > > > gladly resume the work on federated ones. But, it seems to me that
> > > > enabling detached principals is even more urgent today. WDYT?
> > > >
> > > > > I'd like this lookup to be covered by a feature flag [...] it would
> > be
> > > > nice to allow skipping the unnecessary database hit. WDYT?
> > > >
> > > > Yes, that is a very good suggestion!
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > [1]:
> > > >
> > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > > [2]: https://github.com/apache/polaris/pull/1353
> > > >
> > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> > collado.mike@gmail.com>
> > > > wrote:
> > > > >
> > > > > The code change in #1353 prohibited federated principals from being
> > > > > persisted at all. Federated Principal Roles are persisted, but at
> > this
> > > > time
> > > > > (AFAIK), we can't persist federated principals.
> > > > >
> > > > > Mike
> > > > >
> > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <dimas@apache.org
> > >
> > > > wrote:
> > > > >
> > > > > > Hi Alex,
> > > > > >
> > > > > > This proposal sounds very reasonable to me.
> > > > > >
> > > > > > I believe we had some recent user interest (in slack) for running
> > > > Polaris
> > > > > > with Keycloak. From my POV, implementing your proposal should
> > > > > > significantly simplify that user's deployment.
> > > > > >
> > > > > > As for item (3b), I'd like this lookup to be covered by a feature
> > flag.
> > > > > > Admin users who do not import / sync Principals know that this
> > lookup
> > > > will
> > > > > > always fail, so it would be nice to allow skipping the unnecessary
> > > > database
> > > > > > hit. WDYT?
> > > > > >
> > > > > > Thanks,
> > > > > > Dmitri.
> > > > > >
> > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <adutra@apache.org
> > >
> > > > wrote:
> > > > > >
> > > > > > > Hello everyone,
> > > > > > >
> > > > > > > I'd like to propose an enhancement to Polaris's authentication
> > > > > > > process: the introduction of "detached" (non-persisted)
> > principals.
> > > > > > >
> > > > > > > A detached principal is different from a federated principal. The
> > > > > > > former isn't persisted at all, whereas the latter is persisted
> > (with
> > > > > > > federated flag = true), and possibly synchronized with some
> > external
> > > > > > > system (e.g. via SCIM).
> > > > > > >
> > > > > > > Motivation:
> > > > > > >
> > > > > > > Currently, Polaris requires principals to be pre-existing for
> > > > > > > successful authentication – even those marked as "federated". The
> > > > > > > default authenticator always performs a lookup by ID or name, and
> > > > > > > fails if the principal isn't found.
> > > > > > >
> > > > > > > This presents a challenge for users who utilize external
> > > > > > > authentication and authorization systems, such as Keycloak + OPA,
> > > > > > > where Polaris acts as a passthrough between authn and authz.
> > > > > > >
> > > > > > > In such scenarios, principals still need to be created in advance
> > > > > > > within Polaris, which doesn't make much sense.
> > > > > > >
> > > > > > > Proposal:
> > > > > > >
> > > > > > > 1) Introduce a new property: PolarisCredential.isExternal().
> > > > > > >
> > > > > > > 2) During authentication, set isExternal=false on the
> > > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> > otherwise,
> > > > > > > set isExternal=true.
> > > > > > >
> > > > > > > 3) In DefaultAuthenticator:
> > > > > > >
> > > > > > >     a) If PolarisCredential.isExternal() is false, continue with
> > the
> > > > > > > existing logic (lookups by ID and name).
> > > > > > >
> > > > > > >     b) If PolarisCredential.isExternal() is true, perform the
> > usual
> > > > > > > lookups. If they succeed, continue with the existing logic. If
> > they
> > > > > > > fail though, don't fail the authentication. Instead, create the
> > > > > > > principal and assign roles directly from the token.
> > > > > > >
> > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> > re-resolving
> > > > > > > the principal and its roles (as this would fail for detached
> > > > > > > principals).
> > > > > > >
> > > > > > > I believe this change will not impact internal authentication
> > with
> > > > the
> > > > > > > default authorizer. It also won't impact people using external
> > IDPs
> > > > > > > with pre-synchronized federated principals: the default
> > authorizer
> > > > > > > would work just fine.
> > > > > > >
> > > > > > > However, for external IDPs combined with custom authorizers, this
> > > > > > > change would significantly improve the user experience by
> > eliminating
> > > > > > > the need to persist the principal and its roles within Polaris.
> > > > > > >
> > > > > > > What do you all think?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alex
> > > > > > >
> > > > > >
> > > >
> > >
> >

Re: [DISCUSS] "Detached" principals

Posted by Yufei Gu <fl...@gmail.com>.
Thanks for the great discussion. +1 on clarifying the boundary and mutually
exclusive modes, so that there is no confusion and chaos.

Can we have a short design doc to facilitate further discussion?
Particularly. it'd be nice to clarify authN and authZ modes Polaris
supports like the following table:

Mode Principal/Role persistence Privilege enforcement Who decides?
*Native* Persisted Polaris RBAC Polaris
*Federated* Persisted/synced Polaris RBAC Polaris
*Detached* Ephemeral External PDP (e.g. OPA, Ranger) External

Once we clarified different modes, we could start to figure out the
implications and correspondent changes. Things we might consider:
1. Do we allow mixed mode, e.g., some operations are with detached mode,
others are with native mode? I guess this is pretty complex, we may not do
that if there are no strong use cases.
2. How do we handle the audit in case of detached mode? Can we still get
the consistent principal names?

I'm happy to jump on a dedicated call for this.

Yufei


On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <su...@apache.org> wrote:

> Thank you for suggesting this Alex, this is a great discussion.
>
> I see the value of persisted federated roles when Polaris is the primary
> place where catalog privileges are managed and external IdPs just supply
> identity and groups. In that model, stable role entities in Polaris are
> useful and provide a clear governance boundary.
>
> For the OPA-backed PolarisAuthorizer use case, though, we are moving
> toward a setup where:
> - Principals are not persisted in Polaris.
> - Role and attribute evaluation happen in an external Policy Decision
> Point such as OPA.
> - Polaris simply enforces authorization decisions on its REST resources.
>
> In this pattern, persisted federated roles become less compelling and can
> even overlap with external policy configuration. So this “detached
> principal” concept resonates with our use case… it makes the externalized
> responsibility model explicit instead of overloading federated roles to
> serve both the Polaris-managed RBAC and externally-managed Policy Decision
> Point use cases.
>
> So IMHO, it might be worth clarifying the model boundaries so both modes
> are first-class:
> - “Federated” for external identity with locally managed privileges in
> Polaris.
> - “Detached” for external identity with externally managed authorization
> and no local persistence.
>
> That distinction could help keep the semantics clean while still
> minimizing conceptual sprawl.
>
> If it would be helpful, would it make sense to organize a call to discuss
> these concepts for those that are interested? (The usual Polaris Sync time
> slot doesn’t work for me unfortunately, and I’d love to make a case for
> this principal type to be supported)
>
> Cheers,
> Sung
>
> On 2025/11/08 01:09:34 Michael Collado wrote:
> > I see the distinction you're raising. It's a good point, though I'm
> > hesitant to introduce a third principal type to the code base. The most
> > notable thing about federated principals and roles is that role grants
> are
> > never managed through the Polaris API. This would be true even for
> > "detached" principals/roles. Federated principals (in the current code
> > base) also can't be persisted (except possibly through some alternative
> > process, such as SCIM). The main difference is that federated roles are
> > expected to be persisted, but "detached" roles are not. Can we unify
> these
> > concepts so that we can support both ephemeral roles and persisted roles
> > without introducing a third type of role?
> >
> > Mike
> >
> > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> >
> > > Hi all,
> > >
> > > > The code change in #1353 prohibited federated principals from being
> > > persisted at all.
> > >
> > > Indeed! But I think it's important to distinguish between "federated"
> > > and "detached". Here's how I see the difference:
> > >
> > > 1) Federated Principals and Roles
> > >
> > > This approach involves pre-synchronizing principals and roles with an
> > > external system before they can access Polaris. This concept was
> > > outlined in an older design document [1] but never fully implemented.
> > > Typically, this would be achieved using SCIM (eventually) or a
> > > periodic synchronization script run by an administrator.
> > >
> > > Re: federated principals persistence. PR 1353 [2]  assumes
> > > just-in-time (JIT) creation of federated principals, meaning the
> > > principal would be materialized only upon first access. That's not the
> > > case today though. I personally wonder if it would be simpler to also
> > > allow for principal synchronization via the management API. (I'm happy
> > > to resume this conversation later, but that's not the topic here.)
> > >
> > > 2) Detached Principals and Roles
> > >
> > > This scenario implies that no identity management information
> > > whatsoever is stored in Polaris. This fits use cases where the
> > > authorizer does not rely on Polaris's default privilege model: the OPA
> > > authorizer is one example of that. This is the specific use case
> > > currently under discussion.
> > >
> > > Both flavors of principals are important for Polaris, and I would
> > > gladly resume the work on federated ones. But, it seems to me that
> > > enabling detached principals is even more urgent today. WDYT?
> > >
> > > > I'd like this lookup to be covered by a feature flag [...] it would
> be
> > > nice to allow skipping the unnecessary database hit. WDYT?
> > >
> > > Yes, that is a very good suggestion!
> > >
> > > Thanks,
> > > Alex
> > >
> > > [1]:
> > >
> https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > > [2]: https://github.com/apache/polaris/pull/1353
> > >
> > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <
> collado.mike@gmail.com>
> > > wrote:
> > > >
> > > > The code change in #1353 prohibited federated principals from being
> > > > persisted at all. Federated Principal Roles are persisted, but at
> this
> > > time
> > > > (AFAIK), we can't persist federated principals.
> > > >
> > > > Mike
> > > >
> > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <dimas@apache.org
> >
> > > wrote:
> > > >
> > > > > Hi Alex,
> > > > >
> > > > > This proposal sounds very reasonable to me.
> > > > >
> > > > > I believe we had some recent user interest (in slack) for running
> > > Polaris
> > > > > with Keycloak. From my POV, implementing your proposal should
> > > > > significantly simplify that user's deployment.
> > > > >
> > > > > As for item (3b), I'd like this lookup to be covered by a feature
> flag.
> > > > > Admin users who do not import / sync Principals know that this
> lookup
> > > will
> > > > > always fail, so it would be nice to allow skipping the unnecessary
> > > database
> > > > > hit. WDYT?
> > > > >
> > > > > Thanks,
> > > > > Dmitri.
> > > > >
> > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <adutra@apache.org
> >
> > > wrote:
> > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > > I'd like to propose an enhancement to Polaris's authentication
> > > > > > process: the introduction of "detached" (non-persisted)
> principals.
> > > > > >
> > > > > > A detached principal is different from a federated principal. The
> > > > > > former isn't persisted at all, whereas the latter is persisted
> (with
> > > > > > federated flag = true), and possibly synchronized with some
> external
> > > > > > system (e.g. via SCIM).
> > > > > >
> > > > > > Motivation:
> > > > > >
> > > > > > Currently, Polaris requires principals to be pre-existing for
> > > > > > successful authentication – even those marked as "federated". The
> > > > > > default authenticator always performs a lookup by ID or name, and
> > > > > > fails if the principal isn't found.
> > > > > >
> > > > > > This presents a challenge for users who utilize external
> > > > > > authentication and authorization systems, such as Keycloak + OPA,
> > > > > > where Polaris acts as a passthrough between authn and authz.
> > > > > >
> > > > > > In such scenarios, principals still need to be created in advance
> > > > > > within Polaris, which doesn't make much sense.
> > > > > >
> > > > > > Proposal:
> > > > > >
> > > > > > 1) Introduce a new property: PolarisCredential.isExternal().
> > > > > >
> > > > > > 2) During authentication, set isExternal=false on the
> > > > > > PolarisCredential if Polaris is the IDP (internal auth);
> otherwise,
> > > > > > set isExternal=true.
> > > > > >
> > > > > > 3) In DefaultAuthenticator:
> > > > > >
> > > > > >     a) If PolarisCredential.isExternal() is false, continue with
> the
> > > > > > existing logic (lookups by ID and name).
> > > > > >
> > > > > >     b) If PolarisCredential.isExternal() is true, perform the
> usual
> > > > > > lookups. If they succeed, continue with the existing logic. If
> they
> > > > > > fail though, don't fail the authentication. Instead, create the
> > > > > > principal and assign roles directly from the token.
> > > > > >
> > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid
> re-resolving
> > > > > > the principal and its roles (as this would fail for detached
> > > > > > principals).
> > > > > >
> > > > > > I believe this change will not impact internal authentication
> with
> > > the
> > > > > > default authorizer. It also won't impact people using external
> IDPs
> > > > > > with pre-synchronized federated principals: the default
> authorizer
> > > > > > would work just fine.
> > > > > >
> > > > > > However, for external IDPs combined with custom authorizers, this
> > > > > > change would significantly improve the user experience by
> eliminating
> > > > > > the need to persist the principal and its roles within Polaris.
> > > > > >
> > > > > > What do you all think?
> > > > > >
> > > > > > Thanks,
> > > > > > Alex
> > > > > >
> > > > >
> > >
> >
>

Re: [DISCUSS] "Detached" principals

Posted by Sung Yun <su...@apache.org>.
Thank you for suggesting this Alex, this is a great discussion.

I see the value of persisted federated roles when Polaris is the primary place where catalog privileges are managed and external IdPs just supply identity and groups. In that model, stable role entities in Polaris are useful and provide a clear governance boundary.

For the OPA-backed PolarisAuthorizer use case, though, we are moving toward a setup where:
- Principals are not persisted in Polaris.
- Role and attribute evaluation happen in an external Policy Decision Point such as OPA.
- Polaris simply enforces authorization decisions on its REST resources.

In this pattern, persisted federated roles become less compelling and can even overlap with external policy configuration. So this “detached principal” concept resonates with our use case… it makes the externalized responsibility model explicit instead of overloading federated roles to serve both the Polaris-managed RBAC and externally-managed Policy Decision Point use cases.

So IMHO, it might be worth clarifying the model boundaries so both modes are first-class:
- “Federated” for external identity with locally managed privileges in Polaris.
- “Detached” for external identity with externally managed authorization and no local persistence.

That distinction could help keep the semantics clean while still minimizing conceptual sprawl.

If it would be helpful, would it make sense to organize a call to discuss these concepts for those that are interested? (The usual Polaris Sync time slot doesn’t work for me unfortunately, and I’d love to make a case for this principal type to be supported)

Cheers,
Sung

On 2025/11/08 01:09:34 Michael Collado wrote:
> I see the distinction you're raising. It's a good point, though I'm
> hesitant to introduce a third principal type to the code base. The most
> notable thing about federated principals and roles is that role grants are
> never managed through the Polaris API. This would be true even for
> "detached" principals/roles. Federated principals (in the current code
> base) also can't be persisted (except possibly through some alternative
> process, such as SCIM). The main difference is that federated roles are
> expected to be persisted, but "detached" roles are not. Can we unify these
> concepts so that we can support both ephemeral roles and persisted roles
> without introducing a third type of role?
> 
> Mike
> 
> On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org> wrote:
> 
> > Hi all,
> >
> > > The code change in #1353 prohibited federated principals from being
> > persisted at all.
> >
> > Indeed! But I think it's important to distinguish between "federated"
> > and "detached". Here's how I see the difference:
> >
> > 1) Federated Principals and Roles
> >
> > This approach involves pre-synchronizing principals and roles with an
> > external system before they can access Polaris. This concept was
> > outlined in an older design document [1] but never fully implemented.
> > Typically, this would be achieved using SCIM (eventually) or a
> > periodic synchronization script run by an administrator.
> >
> > Re: federated principals persistence. PR 1353 [2]  assumes
> > just-in-time (JIT) creation of federated principals, meaning the
> > principal would be materialized only upon first access. That's not the
> > case today though. I personally wonder if it would be simpler to also
> > allow for principal synchronization via the management API. (I'm happy
> > to resume this conversation later, but that's not the topic here.)
> >
> > 2) Detached Principals and Roles
> >
> > This scenario implies that no identity management information
> > whatsoever is stored in Polaris. This fits use cases where the
> > authorizer does not rely on Polaris's default privilege model: the OPA
> > authorizer is one example of that. This is the specific use case
> > currently under discussion.
> >
> > Both flavors of principals are important for Polaris, and I would
> > gladly resume the work on federated ones. But, it seems to me that
> > enabling detached principals is even more urgent today. WDYT?
> >
> > > I'd like this lookup to be covered by a feature flag [...] it would be
> > nice to allow skipping the unnecessary database hit. WDYT?
> >
> > Yes, that is a very good suggestion!
> >
> > Thanks,
> > Alex
> >
> > [1]:
> > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> > [2]: https://github.com/apache/polaris/pull/1353
> >
> > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <co...@gmail.com>
> > wrote:
> > >
> > > The code change in #1353 prohibited federated principals from being
> > > persisted at all. Federated Principal Roles are persisted, but at this
> > time
> > > (AFAIK), we can't persist federated principals.
> > >
> > > Mike
> > >
> > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> > >
> > > > Hi Alex,
> > > >
> > > > This proposal sounds very reasonable to me.
> > > >
> > > > I believe we had some recent user interest (in slack) for running
> > Polaris
> > > > with Keycloak. From my POV, implementing your proposal should
> > > > significantly simplify that user's deployment.
> > > >
> > > > As for item (3b), I'd like this lookup to be covered by a feature flag.
> > > > Admin users who do not import / sync Principals know that this lookup
> > will
> > > > always fail, so it would be nice to allow skipping the unnecessary
> > database
> > > > hit. WDYT?
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <ad...@apache.org>
> > wrote:
> > > >
> > > > > Hello everyone,
> > > > >
> > > > > I'd like to propose an enhancement to Polaris's authentication
> > > > > process: the introduction of "detached" (non-persisted) principals.
> > > > >
> > > > > A detached principal is different from a federated principal. The
> > > > > former isn't persisted at all, whereas the latter is persisted (with
> > > > > federated flag = true), and possibly synchronized with some external
> > > > > system (e.g. via SCIM).
> > > > >
> > > > > Motivation:
> > > > >
> > > > > Currently, Polaris requires principals to be pre-existing for
> > > > > successful authentication – even those marked as "federated". The
> > > > > default authenticator always performs a lookup by ID or name, and
> > > > > fails if the principal isn't found.
> > > > >
> > > > > This presents a challenge for users who utilize external
> > > > > authentication and authorization systems, such as Keycloak + OPA,
> > > > > where Polaris acts as a passthrough between authn and authz.
> > > > >
> > > > > In such scenarios, principals still need to be created in advance
> > > > > within Polaris, which doesn't make much sense.
> > > > >
> > > > > Proposal:
> > > > >
> > > > > 1) Introduce a new property: PolarisCredential.isExternal().
> > > > >
> > > > > 2) During authentication, set isExternal=false on the
> > > > > PolarisCredential if Polaris is the IDP (internal auth); otherwise,
> > > > > set isExternal=true.
> > > > >
> > > > > 3) In DefaultAuthenticator:
> > > > >
> > > > >     a) If PolarisCredential.isExternal() is false, continue with the
> > > > > existing logic (lookups by ID and name).
> > > > >
> > > > >     b) If PolarisCredential.isExternal() is true, perform the usual
> > > > > lookups. If they succeed, continue with the existing logic. If they
> > > > > fail though, don't fail the authentication. Instead, create the
> > > > > principal and assign roles directly from the token.
> > > > >
> > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
> > > > > the principal and its roles (as this would fail for detached
> > > > > principals).
> > > > >
> > > > > I believe this change will not impact internal authentication with
> > the
> > > > > default authorizer. It also won't impact people using external IDPs
> > > > > with pre-synchronized federated principals: the default authorizer
> > > > > would work just fine.
> > > > >
> > > > > However, for external IDPs combined with custom authorizers, this
> > > > > change would significantly improve the user experience by eliminating
> > > > > the need to persist the principal and its roles within Polaris.
> > > > >
> > > > > What do you all think?
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > >
> >
> 

Re: [DISCUSS] "Detached" principals

Posted by Michael Collado <co...@gmail.com>.
I see the distinction you're raising. It's a good point, though I'm
hesitant to introduce a third principal type to the code base. The most
notable thing about federated principals and roles is that role grants are
never managed through the Polaris API. This would be true even for
"detached" principals/roles. Federated principals (in the current code
base) also can't be persisted (except possibly through some alternative
process, such as SCIM). The main difference is that federated roles are
expected to be persisted, but "detached" roles are not. Can we unify these
concepts so that we can support both ephemeral roles and persisted roles
without introducing a third type of role?

Mike

On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <ad...@apache.org> wrote:

> Hi all,
>
> > The code change in #1353 prohibited federated principals from being
> persisted at all.
>
> Indeed! But I think it's important to distinguish between "federated"
> and "detached". Here's how I see the difference:
>
> 1) Federated Principals and Roles
>
> This approach involves pre-synchronizing principals and roles with an
> external system before they can access Polaris. This concept was
> outlined in an older design document [1] but never fully implemented.
> Typically, this would be achieved using SCIM (eventually) or a
> periodic synchronization script run by an administrator.
>
> Re: federated principals persistence. PR 1353 [2]  assumes
> just-in-time (JIT) creation of federated principals, meaning the
> principal would be materialized only upon first access. That's not the
> case today though. I personally wonder if it would be simpler to also
> allow for principal synchronization via the management API. (I'm happy
> to resume this conversation later, but that's not the topic here.)
>
> 2) Detached Principals and Roles
>
> This scenario implies that no identity management information
> whatsoever is stored in Polaris. This fits use cases where the
> authorizer does not rely on Polaris's default privilege model: the OPA
> authorizer is one example of that. This is the specific use case
> currently under discussion.
>
> Both flavors of principals are important for Polaris, and I would
> gladly resume the work on federated ones. But, it seems to me that
> enabling detached principals is even more urgent today. WDYT?
>
> > I'd like this lookup to be covered by a feature flag [...] it would be
> nice to allow skipping the unnecessary database hit. WDYT?
>
> Yes, that is a very good suggestion!
>
> Thanks,
> Alex
>
> [1]:
> https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
> [2]: https://github.com/apache/polaris/pull/1353
>
> On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <co...@gmail.com>
> wrote:
> >
> > The code change in #1353 prohibited federated principals from being
> > persisted at all. Federated Principal Roles are persisted, but at this
> time
> > (AFAIK), we can't persist federated principals.
> >
> > Mike
> >
> > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
> >
> > > Hi Alex,
> > >
> > > This proposal sounds very reasonable to me.
> > >
> > > I believe we had some recent user interest (in slack) for running
> Polaris
> > > with Keycloak. From my POV, implementing your proposal should
> > > significantly simplify that user's deployment.
> > >
> > > As for item (3b), I'd like this lookup to be covered by a feature flag.
> > > Admin users who do not import / sync Principals know that this lookup
> will
> > > always fail, so it would be nice to allow skipping the unnecessary
> database
> > > hit. WDYT?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <ad...@apache.org>
> wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > I'd like to propose an enhancement to Polaris's authentication
> > > > process: the introduction of "detached" (non-persisted) principals.
> > > >
> > > > A detached principal is different from a federated principal. The
> > > > former isn't persisted at all, whereas the latter is persisted (with
> > > > federated flag = true), and possibly synchronized with some external
> > > > system (e.g. via SCIM).
> > > >
> > > > Motivation:
> > > >
> > > > Currently, Polaris requires principals to be pre-existing for
> > > > successful authentication – even those marked as "federated". The
> > > > default authenticator always performs a lookup by ID or name, and
> > > > fails if the principal isn't found.
> > > >
> > > > This presents a challenge for users who utilize external
> > > > authentication and authorization systems, such as Keycloak + OPA,
> > > > where Polaris acts as a passthrough between authn and authz.
> > > >
> > > > In such scenarios, principals still need to be created in advance
> > > > within Polaris, which doesn't make much sense.
> > > >
> > > > Proposal:
> > > >
> > > > 1) Introduce a new property: PolarisCredential.isExternal().
> > > >
> > > > 2) During authentication, set isExternal=false on the
> > > > PolarisCredential if Polaris is the IDP (internal auth); otherwise,
> > > > set isExternal=true.
> > > >
> > > > 3) In DefaultAuthenticator:
> > > >
> > > >     a) If PolarisCredential.isExternal() is false, continue with the
> > > > existing logic (lookups by ID and name).
> > > >
> > > >     b) If PolarisCredential.isExternal() is true, perform the usual
> > > > lookups. If they succeed, continue with the existing logic. If they
> > > > fail though, don't fail the authentication. Instead, create the
> > > > principal and assign roles directly from the token.
> > > >
> > > > 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
> > > > the principal and its roles (as this would fail for detached
> > > > principals).
> > > >
> > > > I believe this change will not impact internal authentication with
> the
> > > > default authorizer. It also won't impact people using external IDPs
> > > > with pre-synchronized federated principals: the default authorizer
> > > > would work just fine.
> > > >
> > > > However, for external IDPs combined with custom authorizers, this
> > > > change would significantly improve the user experience by eliminating
> > > > the need to persist the principal and its roles within Polaris.
> > > >
> > > > What do you all think?
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > >
>

Re: [DISCUSS] "Detached" principals

Posted by Alexandre Dutra <ad...@apache.org>.
Hi all,

> The code change in #1353 prohibited federated principals from being persisted at all.

Indeed! But I think it's important to distinguish between "federated"
and "detached". Here's how I see the difference:

1) Federated Principals and Roles

This approach involves pre-synchronizing principals and roles with an
external system before they can access Polaris. This concept was
outlined in an older design document [1] but never fully implemented.
Typically, this would be achieved using SCIM (eventually) or a
periodic synchronization script run by an administrator.

Re: federated principals persistence. PR 1353 [2]  assumes
just-in-time (JIT) creation of federated principals, meaning the
principal would be materialized only upon first access. That's not the
case today though. I personally wonder if it would be simpler to also
allow for principal synchronization via the management API. (I'm happy
to resume this conversation later, but that's not the topic here.)

2) Detached Principals and Roles

This scenario implies that no identity management information
whatsoever is stored in Polaris. This fits use cases where the
authorizer does not rely on Polaris's default privilege model: the OPA
authorizer is one example of that. This is the specific use case
currently under discussion.

Both flavors of principals are important for Polaris, and I would
gladly resume the work on federated ones. But, it seems to me that
enabling detached principals is even more urgent today. WDYT?

> I'd like this lookup to be covered by a feature flag [...] it would be nice to allow skipping the unnecessary database hit. WDYT?

Yes, that is a very good suggestion!

Thanks,
Alex

[1]: https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5
[2]: https://github.com/apache/polaris/pull/1353

On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <co...@gmail.com> wrote:
>
> The code change in #1353 prohibited federated principals from being
> persisted at all. Federated Principal Roles are persisted, but at this time
> (AFAIK), we can't persist federated principals.
>
> Mike
>
> On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> > Hi Alex,
> >
> > This proposal sounds very reasonable to me.
> >
> > I believe we had some recent user interest (in slack) for running Polaris
> > with Keycloak. From my POV, implementing your proposal should
> > significantly simplify that user's deployment.
> >
> > As for item (3b), I'd like this lookup to be covered by a feature flag.
> > Admin users who do not import / sync Principals know that this lookup will
> > always fail, so it would be nice to allow skipping the unnecessary database
> > hit. WDYT?
> >
> > Thanks,
> > Dmitri.
> >
> > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <ad...@apache.org> wrote:
> >
> > > Hello everyone,
> > >
> > > I'd like to propose an enhancement to Polaris's authentication
> > > process: the introduction of "detached" (non-persisted) principals.
> > >
> > > A detached principal is different from a federated principal. The
> > > former isn't persisted at all, whereas the latter is persisted (with
> > > federated flag = true), and possibly synchronized with some external
> > > system (e.g. via SCIM).
> > >
> > > Motivation:
> > >
> > > Currently, Polaris requires principals to be pre-existing for
> > > successful authentication – even those marked as "federated". The
> > > default authenticator always performs a lookup by ID or name, and
> > > fails if the principal isn't found.
> > >
> > > This presents a challenge for users who utilize external
> > > authentication and authorization systems, such as Keycloak + OPA,
> > > where Polaris acts as a passthrough between authn and authz.
> > >
> > > In such scenarios, principals still need to be created in advance
> > > within Polaris, which doesn't make much sense.
> > >
> > > Proposal:
> > >
> > > 1) Introduce a new property: PolarisCredential.isExternal().
> > >
> > > 2) During authentication, set isExternal=false on the
> > > PolarisCredential if Polaris is the IDP (internal auth); otherwise,
> > > set isExternal=true.
> > >
> > > 3) In DefaultAuthenticator:
> > >
> > >     a) If PolarisCredential.isExternal() is false, continue with the
> > > existing logic (lookups by ID and name).
> > >
> > >     b) If PolarisCredential.isExternal() is true, perform the usual
> > > lookups. If they succeed, continue with the existing logic. If they
> > > fail though, don't fail the authentication. Instead, create the
> > > principal and assign roles directly from the token.
> > >
> > > 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
> > > the principal and its roles (as this would fail for detached
> > > principals).
> > >
> > > I believe this change will not impact internal authentication with the
> > > default authorizer. It also won't impact people using external IDPs
> > > with pre-synchronized federated principals: the default authorizer
> > > would work just fine.
> > >
> > > However, for external IDPs combined with custom authorizers, this
> > > change would significantly improve the user experience by eliminating
> > > the need to persist the principal and its roles within Polaris.
> > >
> > > What do you all think?
> > >
> > > Thanks,
> > > Alex
> > >
> >

Re: [DISCUSS] "Detached" principals

Posted by Michael Collado <co...@gmail.com>.
The code change in #1353 prohibited federated principals from being
persisted at all. Federated Principal Roles are persisted, but at this time
(AFAIK), we can't persist federated principals.

Mike

On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <di...@apache.org> wrote:

> Hi Alex,
>
> This proposal sounds very reasonable to me.
>
> I believe we had some recent user interest (in slack) for running Polaris
> with Keycloak. From my POV, implementing your proposal should
> significantly simplify that user's deployment.
>
> As for item (3b), I'd like this lookup to be covered by a feature flag.
> Admin users who do not import / sync Principals know that this lookup will
> always fail, so it would be nice to allow skipping the unnecessary database
> hit. WDYT?
>
> Thanks,
> Dmitri.
>
> On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <ad...@apache.org> wrote:
>
> > Hello everyone,
> >
> > I'd like to propose an enhancement to Polaris's authentication
> > process: the introduction of "detached" (non-persisted) principals.
> >
> > A detached principal is different from a federated principal. The
> > former isn't persisted at all, whereas the latter is persisted (with
> > federated flag = true), and possibly synchronized with some external
> > system (e.g. via SCIM).
> >
> > Motivation:
> >
> > Currently, Polaris requires principals to be pre-existing for
> > successful authentication – even those marked as "federated". The
> > default authenticator always performs a lookup by ID or name, and
> > fails if the principal isn't found.
> >
> > This presents a challenge for users who utilize external
> > authentication and authorization systems, such as Keycloak + OPA,
> > where Polaris acts as a passthrough between authn and authz.
> >
> > In such scenarios, principals still need to be created in advance
> > within Polaris, which doesn't make much sense.
> >
> > Proposal:
> >
> > 1) Introduce a new property: PolarisCredential.isExternal().
> >
> > 2) During authentication, set isExternal=false on the
> > PolarisCredential if Polaris is the IDP (internal auth); otherwise,
> > set isExternal=true.
> >
> > 3) In DefaultAuthenticator:
> >
> >     a) If PolarisCredential.isExternal() is false, continue with the
> > existing logic (lookups by ID and name).
> >
> >     b) If PolarisCredential.isExternal() is true, perform the usual
> > lookups. If they succeed, continue with the existing logic. If they
> > fail though, don't fail the authentication. Instead, create the
> > principal and assign roles directly from the token.
> >
> > 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
> > the principal and its roles (as this would fail for detached
> > principals).
> >
> > I believe this change will not impact internal authentication with the
> > default authorizer. It also won't impact people using external IDPs
> > with pre-synchronized federated principals: the default authorizer
> > would work just fine.
> >
> > However, for external IDPs combined with custom authorizers, this
> > change would significantly improve the user experience by eliminating
> > the need to persist the principal and its roles within Polaris.
> >
> > What do you all think?
> >
> > Thanks,
> > Alex
> >
>

Re: [DISCUSS] "Detached" principals

Posted by Dmitri Bourlatchkov <di...@apache.org>.
Hi Alex,

This proposal sounds very reasonable to me.

I believe we had some recent user interest (in slack) for running Polaris
with Keycloak. From my POV, implementing your proposal should
significantly simplify that user's deployment.

As for item (3b), I'd like this lookup to be covered by a feature flag.
Admin users who do not import / sync Principals know that this lookup will
always fail, so it would be nice to allow skipping the unnecessary database
hit. WDYT?

Thanks,
Dmitri.

On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <ad...@apache.org> wrote:

> Hello everyone,
>
> I'd like to propose an enhancement to Polaris's authentication
> process: the introduction of "detached" (non-persisted) principals.
>
> A detached principal is different from a federated principal. The
> former isn't persisted at all, whereas the latter is persisted (with
> federated flag = true), and possibly synchronized with some external
> system (e.g. via SCIM).
>
> Motivation:
>
> Currently, Polaris requires principals to be pre-existing for
> successful authentication – even those marked as "federated". The
> default authenticator always performs a lookup by ID or name, and
> fails if the principal isn't found.
>
> This presents a challenge for users who utilize external
> authentication and authorization systems, such as Keycloak + OPA,
> where Polaris acts as a passthrough between authn and authz.
>
> In such scenarios, principals still need to be created in advance
> within Polaris, which doesn't make much sense.
>
> Proposal:
>
> 1) Introduce a new property: PolarisCredential.isExternal().
>
> 2) During authentication, set isExternal=false on the
> PolarisCredential if Polaris is the IDP (internal auth); otherwise,
> set isExternal=true.
>
> 3) In DefaultAuthenticator:
>
>     a) If PolarisCredential.isExternal() is false, continue with the
> existing logic (lookups by ID and name).
>
>     b) If PolarisCredential.isExternal() is true, perform the usual
> lookups. If they succeed, continue with the existing logic. If they
> fail though, don't fail the authentication. Instead, create the
> principal and assign roles directly from the token.
>
> 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving
> the principal and its roles (as this would fail for detached
> principals).
>
> I believe this change will not impact internal authentication with the
> default authorizer. It also won't impact people using external IDPs
> with pre-synchronized federated principals: the default authorizer
> would work just fine.
>
> However, for external IDPs combined with custom authorizers, this
> change would significantly improve the user experience by eliminating
> the need to persist the principal and its roles within Polaris.
>
> What do you all think?
>
> Thanks,
> Alex
>