[DISCUSS] Usefulness of InMemoryEntityCache

[Dmitri Bourlatchkov <di...@apache.org>]

Hi All,

InMemoryEntityCache is currently used in Apache Polaris with JDBC
Persistence.

However, as far as I can tell, even for cached entities, the resolve will
still issue a JDBC query to confirm that the latest version was returned
from the cache.

So, the question is: does this cache provide any considerable efficiency
gain?

Has anyone measured the performance difference with and without this cache?

Would anyone be willing to carry out such a test?

WDYT?

Thanks,
Dmitri.

Re: [DISCUSS] Usefulness of InMemoryEntityCache

[Robert Stupp <sn...@snazy.de>]

Thanks, this makes the warm-cache benefit clear.

Still, 23 SELECTs for a cold resolve of a simple N1/N2/T2 path seems like a
high baseline. Is that expected to remain the normal cold-path shape, or can
the needed hierarchy, grants, and versions be obtained in a bounded number
of
backend operations?

I agree that a multi-pod deployment needs a version/conflict fence. For a
mutation, though, could that normally be the final conditional write, e.g.
UPDATE ... WHERE version = ?, rather than a separate version query on every
cache hit? A zero-row update would identify the conflict, and only that case
would need a reload.

The remaining question is the scope of that fence. A table operation can
depend
on inherited namespace/catalog facts, including effective storage
configuration,
locations, and allowed locations. If one of those changes after resolution,
how
is that detected before credential vending or commit? Checking only the
table
version would not cover an ancestor change.

Could we define that current-state/commit boundary first, and then measure
cold,
warm, and high-churn behavior against it? The 64 KiB property example seems
like a related but separate reason to keep common resolution paths bounded.

On Fri, Sep 4, 2026 at 6:56 PM Yufei Gu <fl...@gmail.com> wrote:

> +1 to Prithvi’s analysis. The version check is necessary to ensure
> consistency in a multi-pod Polaris deployment, since an individual pod
> cannot see entity updates made by its peers.
>
> Yufei
>
>
> On Fri, Sep 4, 2026 at 9:46 AM Prithvi S <pr...@gmail.com>
> wrote:
>
> > Hi Dmitri,
> >
> > Thanks for raising this.
> >
> > You are right that even on a cache hit, resolve still talks to JDBC. That
> > is by design: InMemoryEntityCache is not a "skip the database" cache. For
> > JDBC it is a "skip the expensive load" cache, with a version check as the
> > invalidation path.
> > I traced Resolver.resolveAll() with JDBC and compared a warmed
> > InMemoryEntityCache against cache == null on the same catalog. The
> workload
> > was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL,
> catalog
> > "test", path N1/N2/T2).
> >
> > On a warm cache, that resolve issued 1 SELECT:
> >
> > SELECT id, catalog_id, entity_version, grant_records_version
> > FROM ENTITIES
> > WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ?
> >
> > Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows
> > (including properties / internal_properties) and 14 grant-record lookups.
> > Scenario Cache SELECTs Payload
> > N1/N2/T2 (small properties) no-cache 23 947 B
> > N1/N2/T2 cold-cache 23 947 B
> > N1/N2/T2 warm-cache 1 144 B
> > N1/N2/T1 (64KiB properties) no-cache 23 66 KB
> > N1/N2/T1 warm-cache 1 144 B
> > deep path (6 segments) no-cache 29 959 B
> > deep path (6 segments) warm-cache 1 192 B
> >
> > So the version check is real, but it replaces the full entity JSON and
> the
> > grant-record queries. Cold cache looks the same as no cache; the gain is
> on
> > subsequent resolves of the same entities.
> >
> > I have not measured a full REST request or a stale-cache / high-churn
> path,
> > so I do not want to claim an end-to-end speedup from this :)
> > For the resolve path itself, the query shape is clear.
> >
> > WDYT?
> >
> > Thanks,
> > Prithvi
> >
> > On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> >
> > > Hi All,
> > >
> > > InMemoryEntityCache is currently used in Apache Polaris with JDBC
> > > Persistence.
> > >
> > > However, as far as I can tell, even for cached entities, the resolve
> will
> > > still issue a JDBC query to confirm that the latest version was
> returned
> > > from the cache.
> > >
> > > So, the question is: does this cache provide any considerable
> efficiency
> > > gain?
> > >
> > > Has anyone measured the performance difference with and without this
> > cache?
> > >
> > > Would anyone be willing to carry out such a test?
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> >
>

Re: [DISCUSS] Usefulness of InMemoryEntityCache

[Yufei Gu <fl...@gmail.com>]

+1 to Prithvi’s analysis. The version check is necessary to ensure
consistency in a multi-pod Polaris deployment, since an individual pod
cannot see entity updates made by its peers.

Yufei


On Fri, Sep 4, 2026 at 9:46 AM Prithvi S <pr...@gmail.com>
wrote:

> Hi Dmitri,
>
> Thanks for raising this.
>
> You are right that even on a cache hit, resolve still talks to JDBC. That
> is by design: InMemoryEntityCache is not a "skip the database" cache. For
> JDBC it is a "skip the expensive load" cache, with a version check as the
> invalidation path.
> I traced Resolver.resolveAll() with JDBC and compared a warmed
> InMemoryEntityCache against cache == null on the same catalog. The workload
> was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL, catalog
> "test", path N1/N2/T2).
>
> On a warm cache, that resolve issued 1 SELECT:
>
> SELECT id, catalog_id, entity_version, grant_records_version
> FROM ENTITIES
> WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ?
>
> Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows
> (including properties / internal_properties) and 14 grant-record lookups.
> Scenario Cache SELECTs Payload
> N1/N2/T2 (small properties) no-cache 23 947 B
> N1/N2/T2 cold-cache 23 947 B
> N1/N2/T2 warm-cache 1 144 B
> N1/N2/T1 (64KiB properties) no-cache 23 66 KB
> N1/N2/T1 warm-cache 1 144 B
> deep path (6 segments) no-cache 29 959 B
> deep path (6 segments) warm-cache 1 192 B
>
> So the version check is real, but it replaces the full entity JSON and the
> grant-record queries. Cold cache looks the same as no cache; the gain is on
> subsequent resolves of the same entities.
>
> I have not measured a full REST request or a stale-cache / high-churn path,
> so I do not want to claim an end-to-end speedup from this :)
> For the resolve path itself, the query shape is clear.
>
> WDYT?
>
> Thanks,
> Prithvi
>
> On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
> > Hi All,
> >
> > InMemoryEntityCache is currently used in Apache Polaris with JDBC
> > Persistence.
> >
> > However, as far as I can tell, even for cached entities, the resolve will
> > still issue a JDBC query to confirm that the latest version was returned
> > from the cache.
> >
> > So, the question is: does this cache provide any considerable efficiency
> > gain?
> >
> > Has anyone measured the performance difference with and without this
> cache?
> >
> > Would anyone be willing to carry out such a test?
> >
> > WDYT?
> >
> > Thanks,
> > Dmitri.
> >
>

Re: [DISCUSS] Usefulness of InMemoryEntityCache

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Hi guys,

I think Prithvi and Yufei have the right framing. My take is that the
cache's job is to skip expensive grant-record load, not really to skip
the database entirely (that's a larger concern). And that's the
purpose to any cache that must stay consistent across pods.
However, Robert's complaint about 23 cold-path queries is legitimate
and traces the "real" code:
AtomicOperationMetaStoreManager.loadResolvedEntityById() issues 1 or 2
grant-record queries per entity, and it's called once per entity in a
resolved path with no batching. That's N+1, not related to caching
cost.
Imho, we have independent fixes here.
I like Dmitri's idea about skipping grant-record lookups entirely when
an external authorizer (Ranger) owns authorization, but I believe it
would need extra plumbing to know "no internal grants needed" at the
persistence layer, else I suspect we will load the persistence layer
anyway.

I might be wrong, but:
1. I think grant-record loading is never actually batched, even in the
"batch" API ;).
BasePersistence.loadAllGrantRecordsOnSecurable()/loadAllGrantRecordsOnGrantee()
only take a single entity id.
AtomicOperationMetaStoreManager.loadResolvedEntities() does batch the
entity-row lookup via lookupEntities(), but then
toResolvedPolarisEntity() loops over each entity calling the
single-entity grant methods. So, if I do the math, a "batched"
5-entity load still fires up to 10 grant queries. Same story in the
single-entity loadResolvedEntityById() used by the cold-cache-miss
path. I believe we can fix that with a JDBC composite-key IN query.
The good news is that QueryGenerator already has the template for this
(generateSelectQueryWithEntityIds()): a
loadAllGrantRecordsOnSecurables(List<PolarisEntityId>) variant against
ModelGrantRecord would be a direct copy of that pattern.
2. The Resolver.bulkValidate() has a batched version-check.
bulkValidate() does one batched loadEntitiesChangeTracking call to get
a fresh version for the whole toValidate set. But for every entity
that's actually stale, it calls cache.getAndRefreshIfNeeded() one at a
time, and that method calls refreshResolvedEntity(), which re-runs
lookupEntityVersions() again per entity :/

I would suggest to evaluate how we can already improve that. I'm quite
sure bulkValidate() refresh branch can be optimized (probably using
the existing getOrLoadResolvedEntitties()).

So before going to deep in cache refactoring, I think we can already
do nice improvements.

Regards
JB

On Tue, Sep 8, 2026 at 4:23 PM Dmitri Bourlatchkov <di...@apache.org> wrote:
>
> Hi Prithvi,
>
> Thanks for the analysis! It is very helpful.
>
> The grant record lookups appear to be specific to the internal RBAC model.
> I wonder if they are necessary with external Authorizers (e.g. Ranger). I
> suppose those queries will return empty results in the external case anyway.
>
> What do people think about refactoring the resolution code path to take the
> Authorizer "kind" into account? I believe Sung's prior work was moving in
> that direction. After PR [5194] we should be able to avoid the grant record
> lookup for authorizers that do not use them, I hope.
>
> [5194] https://github.com/apache/polaris/pull/5194
>
> Cheers,
> Dmitri.
>
> On Fri, Sep 4, 2026 at 12:46 PM Prithvi S <pr...@gmail.com>
> wrote:
>
> > Hi Dmitri,
> >
> > Thanks for raising this.
> >
> > You are right that even on a cache hit, resolve still talks to JDBC. That
> > is by design: InMemoryEntityCache is not a "skip the database" cache. For
> > JDBC it is a "skip the expensive load" cache, with a version check as the
> > invalidation path.
> > I traced Resolver.resolveAll() with JDBC and compared a warmed
> > InMemoryEntityCache against cache == null on the same catalog. The workload
> > was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL, catalog
> > "test", path N1/N2/T2).
> >
> > On a warm cache, that resolve issued 1 SELECT:
> >
> > SELECT id, catalog_id, entity_version, grant_records_version
> > FROM ENTITIES
> > WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ?
> >
> > Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows
> > (including properties / internal_properties) and 14 grant-record lookups.
> > Scenario Cache SELECTs Payload
> > N1/N2/T2 (small properties) no-cache 23 947 B
> > N1/N2/T2 cold-cache 23 947 B
> > N1/N2/T2 warm-cache 1 144 B
> > N1/N2/T1 (64KiB properties) no-cache 23 66 KB
> > N1/N2/T1 warm-cache 1 144 B
> > deep path (6 segments) no-cache 29 959 B
> > deep path (6 segments) warm-cache 1 192 B
> >
> > So the version check is real, but it replaces the full entity JSON and the
> > grant-record queries. Cold cache looks the same as no cache; the gain is on
> > subsequent resolves of the same entities.
> >
> > I have not measured a full REST request or a stale-cache / high-churn path,
> > so I do not want to claim an end-to-end speedup from this :)
> > For the resolve path itself, the query shape is clear.
> >
> > WDYT?
> >
> > Thanks,
> > Prithvi
> >
> > On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <di...@apache.org>
> > wrote:
> >
> > > Hi All,
> > >
> > > InMemoryEntityCache is currently used in Apache Polaris with JDBC
> > > Persistence.
> > >
> > > However, as far as I can tell, even for cached entities, the resolve will
> > > still issue a JDBC query to confirm that the latest version was returned
> > > from the cache.
> > >
> > > So, the question is: does this cache provide any considerable efficiency
> > > gain?
> > >
> > > Has anyone measured the performance difference with and without this
> > cache?
> > >
> > > Would anyone be willing to carry out such a test?
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Dmitri.
> > >
> >

Re: [DISCUSS] Usefulness of InMemoryEntityCache

[Dmitri Bourlatchkov <di...@apache.org>]

Hi Prithvi,

Thanks for the analysis! It is very helpful.

The grant record lookups appear to be specific to the internal RBAC model.
I wonder if they are necessary with external Authorizers (e.g. Ranger). I
suppose those queries will return empty results in the external case anyway.

What do people think about refactoring the resolution code path to take the
Authorizer "kind" into account? I believe Sung's prior work was moving in
that direction. After PR [5194] we should be able to avoid the grant record
lookup for authorizers that do not use them, I hope.

[5194] https://github.com/apache/polaris/pull/5194

Cheers,
Dmitri.

On Fri, Sep 4, 2026 at 12:46 PM Prithvi S <pr...@gmail.com>
wrote:

> Hi Dmitri,
>
> Thanks for raising this.
>
> You are right that even on a cache hit, resolve still talks to JDBC. That
> is by design: InMemoryEntityCache is not a "skip the database" cache. For
> JDBC it is a "skip the expensive load" cache, with a version check as the
> invalidation path.
> I traced Resolver.resolveAll() with JDBC and compared a warmed
> InMemoryEntityCache against cache == null on the same catalog. The workload
> was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL, catalog
> "test", path N1/N2/T2).
>
> On a warm cache, that resolve issued 1 SELECT:
>
> SELECT id, catalog_id, entity_version, grant_records_version
> FROM ENTITIES
> WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ?
>
> Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows
> (including properties / internal_properties) and 14 grant-record lookups.
> Scenario Cache SELECTs Payload
> N1/N2/T2 (small properties) no-cache 23 947 B
> N1/N2/T2 cold-cache 23 947 B
> N1/N2/T2 warm-cache 1 144 B
> N1/N2/T1 (64KiB properties) no-cache 23 66 KB
> N1/N2/T1 warm-cache 1 144 B
> deep path (6 segments) no-cache 29 959 B
> deep path (6 segments) warm-cache 1 192 B
>
> So the version check is real, but it replaces the full entity JSON and the
> grant-record queries. Cold cache looks the same as no cache; the gain is on
> subsequent resolves of the same entities.
>
> I have not measured a full REST request or a stale-cache / high-churn path,
> so I do not want to claim an end-to-end speedup from this :)
> For the resolve path itself, the query shape is clear.
>
> WDYT?
>
> Thanks,
> Prithvi
>
> On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
> > Hi All,
> >
> > InMemoryEntityCache is currently used in Apache Polaris with JDBC
> > Persistence.
> >
> > However, as far as I can tell, even for cached entities, the resolve will
> > still issue a JDBC query to confirm that the latest version was returned
> > from the cache.
> >
> > So, the question is: does this cache provide any considerable efficiency
> > gain?
> >
> > Has anyone measured the performance difference with and without this
> cache?
> >
> > Would anyone be willing to carry out such a test?
> >
> > WDYT?
> >
> > Thanks,
> > Dmitri.
> >
>

Re: [DISCUSS] Usefulness of InMemoryEntityCache

[Prithvi S <pr...@gmail.com>]

Hi Dmitri,

Thanks for raising this.

You are right that even on a cache hit, resolve still talks to JDBC. That
is by design: InMemoryEntityCache is not a "skip the database" cache. For
JDBC it is a "skip the expensive load" cache, with a version check as the
invalidation path.
I traced Resolver.resolveAll() with JDBC and compared a warmed
InMemoryEntityCache against cache == null on the same catalog. The workload
was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL, catalog
"test", path N1/N2/T2).

On a warm cache, that resolve issued 1 SELECT:

SELECT id, catalog_id, entity_version, grant_records_version
FROM ENTITIES
WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ?

Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows
(including properties / internal_properties) and 14 grant-record lookups.
Scenario Cache SELECTs Payload
N1/N2/T2 (small properties) no-cache 23 947 B
N1/N2/T2 cold-cache 23 947 B
N1/N2/T2 warm-cache 1 144 B
N1/N2/T1 (64KiB properties) no-cache 23 66 KB
N1/N2/T1 warm-cache 1 144 B
deep path (6 segments) no-cache 29 959 B
deep path (6 segments) warm-cache 1 192 B

So the version check is real, but it replaces the full entity JSON and the
grant-record queries. Cold cache looks the same as no cache; the gain is on
subsequent resolves of the same entities.

I have not measured a full REST request or a stale-cache / high-churn path,
so I do not want to claim an end-to-end speedup from this :)
For the resolve path itself, the query shape is clear.

WDYT?

Thanks,
Prithvi

On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <di...@apache.org> wrote:

> Hi All,
>
> InMemoryEntityCache is currently used in Apache Polaris with JDBC
> Persistence.
>
> However, as far as I can tell, even for cached entities, the resolve will
> still issue a JDBC query to confirm that the latest version was returned
> from the cache.
>
> So, the question is: does this cache provide any considerable efficiency
> gain?
>
> Has anyone measured the performance difference with and without this cache?
>
> Would anyone be willing to carry out such a test?
>
> WDYT?
>
> Thanks,
> Dmitri.
>