You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "ayushtkn (via GitHub)" <gi...@apache.org> on 2026/07/23 18:28:03 UTC
[PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
ayushtkn opened a new pull request, #5134:
URL: https://github.com/apache/polaris/pull/5134
On the create path, `JdbcBasePersistenceImpl.writeEntities` checked whether each entity already existed (for idempotent retries) by calling `lookupEntity`, which fetches the entire row โ including the large JSON `properties/internal_properties` blobs โ only to discard everything but the fact that a row exists. This replaces that with a cheap `SELECT 1 โฆ LIMIT 1` existence probe (`entityExists`) issued on the transaction connection, so a retried create stays idempotent without pulling back full entities; the `update` path (where the original entity is already known) skips the check entirely. Behavior is unchanged โ only the query cost drops. Includes tests asserting the create path issues an existence-only query (no property columns) and the update path issues no pre-lookup at all.
## Checklist
- [ ] ๐ก๏ธ Don't disclose security issues! (contact security@apache.org)
- [ ] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes #
- [ ] ๐งช Added/updated tests with good coverage, or manually tested (and explained how)
- [ ] ๐ก Added comments for complex logic
- [ ] ๐งพ Updated `CHANGELOG.md` (if needed)
- [ ] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain merged PR #5134:
URL: https://github.com/apache/polaris/pull/5134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #5134:
URL: https://github.com/apache/polaris/pull/5134#discussion_r3648925565
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java:
##########
@@ -143,15 +159,13 @@ public void writeEntities(
PolarisBaseEntity entity = entities.get(i);
PolarisBaseEntity originalEntity =
originalEntities != null ? originalEntities.get(i) : null;
- // first, check if the entity has already been created, in which case we will simply
- // return it.
- PolarisBaseEntity entityFound =
- lookupEntity(
- callCtx, entity.getCatalogId(), entity.getId(), entity.getTypeCode());
- if (entityFound != null && originalEntity == null) {
- // probably the client retried, simply return it
- // TODO: Check correctness of returning entityFound vs entity here. It may have
- // already been updated after the creation.
+ // On the create path (no known original entity) a client retry may be replaying a
+ // create that already succeeded; skip it so the write stays idempotent. Only
+ // existence matters here, so issue a cheap EXISTS check instead of fetching the full
+ // row (including the large JSON property blobs). On the update path the original
+ // entity is already known, so this check is unnecessary and is skipped entirely.
Review Comment:
Nit: I think these comments are unnecessary. Sorry, didn't catch it in the first round.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #5134:
URL: https://github.com/apache/polaris/pull/5134#discussion_r3648100089
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java:
##########
@@ -415,11 +452,20 @@ public void deleteAll(@NonNull PolarisCallContext callCtx) {
@Override
public PolarisBaseEntity lookupEntity(
@NonNull PolarisCallContext callCtx, long catalogId, long entityId, int typeCode) {
- Map<String, Object> params =
- Map.of("catalog_id", catalogId, "id", entityId, "type_code", typeCode, "realm_id", realmId);
return getPolarisBaseEntity(
QueryGenerator.generateSelectQuery(
- ModelEntity.getAllColumnNames(schemaVersion), ModelEntity.TABLE_NAME, params));
+ ModelEntity.getAllColumnNames(schemaVersion),
+ ModelEntity.TABLE_NAME,
+ entityKeyParams(catalogId, entityId, typeCode)));
+ }
+
+ /** WHERE-clause parameters that uniquely identify a single entity row within this realm. */
+ private Map<String, Object> entityKeyParams(long catalogId, long entityId, int typeCode) {
+ return Map.of(
+ "catalog_id", catalogId,
+ "id", entityId,
+ "type_code", typeCode,
+ "realm_id", realmId);
Review Comment:
Do we need catalog id and type code given the following entity table primary key?
```
PRIMARY KEY (realm_id, id)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #5134:
URL: https://github.com/apache/polaris/pull/5134#discussion_r3648670729
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java:
##########
@@ -415,11 +452,20 @@ public void deleteAll(@NonNull PolarisCallContext callCtx) {
@Override
public PolarisBaseEntity lookupEntity(
@NonNull PolarisCallContext callCtx, long catalogId, long entityId, int typeCode) {
- Map<String, Object> params =
- Map.of("catalog_id", catalogId, "id", entityId, "type_code", typeCode, "realm_id", realmId);
return getPolarisBaseEntity(
QueryGenerator.generateSelectQuery(
- ModelEntity.getAllColumnNames(schemaVersion), ModelEntity.TABLE_NAME, params));
+ ModelEntity.getAllColumnNames(schemaVersion),
+ ModelEntity.TABLE_NAME,
+ entityKeyParams(catalogId, entityId, typeCode)));
+ }
+
+ /** WHERE-clause parameters that uniquely identify a single entity row within this realm. */
+ private Map<String, Object> entityKeyParams(long catalogId, long entityId, int typeCode) {
+ return Map.of(
+ "catalog_id", catalogId,
+ "id", entityId,
+ "type_code", typeCode,
+ "realm_id", realmId);
Review Comment:
Thanx @flyrain, have made the suggested change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix JDBC writeEntities issuing a wasteful full-row lookup per entity [polaris]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #5134:
URL: https://github.com/apache/polaris/pull/5134#discussion_r3648990706
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java:
##########
@@ -143,15 +159,13 @@ public void writeEntities(
PolarisBaseEntity entity = entities.get(i);
PolarisBaseEntity originalEntity =
originalEntities != null ? originalEntities.get(i) : null;
- // first, check if the entity has already been created, in which case we will simply
- // return it.
- PolarisBaseEntity entityFound =
- lookupEntity(
- callCtx, entity.getCatalogId(), entity.getId(), entity.getTypeCode());
- if (entityFound != null && originalEntity == null) {
- // probably the client retried, simply return it
- // TODO: Check correctness of returning entityFound vs entity here. It may have
- // already been updated after the creation.
+ // On the create path (no known original entity) a client retry may be replaying a
+ // create that already succeeded; skip it so the write stays idempotent. Only
+ // existence matters here, so issue a cheap EXISTS check instead of fetching the full
+ // row (including the large JSON property blobs). On the update path the original
+ // entity is already known, so this check is unnecessary and is skipped entirely.
Review Comment:
Removed :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org