You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "sririshindra (via GitHub)" <gi...@apache.org> on 2026/02/06 08:30:59 UTC
[PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
sririshindra opened a new pull request, #3682:
URL: https://github.com/apache/polaris/pull/3682
Add support for the new overwrite boolean on RegisterTableRequest (default: false) so clients can register a metadata location that replaces an existing table pointer. Implement register-table overwrite semantics:
If overwrite=false (default): preserve existing behavior โ attempting to register a table that already exists returns a conflict/error. If overwrite=true:
If the table does not exist: create it (normal register). If the table exists: update the table's metadata-location to the provided value (do not throw AlreadyExists). Ensure safety/backward-compatibility:
overwrite defaults to false so existing clients are unaffected. Validate provided metadata location where possible before committing to avoid corrupting catalog state.
Details / rationale:
The change implements the REST Catalog spec extension that adds an overwrite flag to RegisterTableRequest. This enables clients to atomically point an existing table identifier at a new metadata file (useful for moving or restoring table metadata). Because overwrite changes the set of operations allowed on an existing table, we had to be careful with authorization and entity resolution: When performing an overwrite against an existing table, the handler enforces the UPDATE_TABLE privilege (mapped to TABLE_WRITE_PROPERTIES) rather than REGISTER_TABLE. This prevents callers who only have create permissions from silently replacing another principal's table pointer. We fixed a subtle distinction where lack of read privileges could be mistaken for a non-existent table. The handler now distinguishes "truly not found" from "exists but unreadable" and applies the correct required privilege accordingly. Resolution manifest and passthrough-path population were adjusted so downstream overwri
te logic can locate the namespace and table entries reliably. Files / components touched (high level):
Tests added/updated:
- DTO unit tests: overwrite deserialization for `RegisterTableRequest` (true/false/missing).
- Integration / behavior tests (integration-tests module):
- Register new table with `overwrite=false` โ success.
- Register existing table with `overwrite=false` โ conflict / exception as before.
- Register existing table with `overwrite=true` โ success and the table's metadata-location is updated atomically.
- Authorization tests: updated/added unit tests to assert `UPDATE_TABLE` (TABLE_WRITE_PROPERTIES) is enforced for overwrite against existing tables.
- Unit tests: added `CatalogHandlerUtilsTest` to cover handler helper logic.
<!--
๐ Describe what changes you're proposing, especially breaking or user-facing changes.
๐ See https://github.com/apache/polaris/blob/main/CONTRIBUTING.md for more.
-->
## Checklist
- [ ] ๐ก๏ธ Don't disclose security issues! (contact security@apache.org)
- [ ] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes #
- [ ] ๐งช Added/updated tests with good coverage, or manually tested (and explained how)
- [ ] ๐ก Added comments for complex logic
- [ ] ๐งพ Updated `CHANGELOG.md` (if needed)
- [ ] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3862345341
> I have an additional question. My understanding is that once your PR in the iceberg repository gets merged and released as part of the next Iceberg update. We would have to update https://github.com/apache/polaris/blob/main/gradle/libs.versions.toml#L23 to pick up that artifact. Correct?
You are correct in saying that this feature will have to wait for Iceberg 1.11.
But development can happen ahead of schedule! I just created the `feature/iceberg-1.11` branch.
Once the overwrite parameter is merged in Iceberg you would be able to rebase your current work on top of that branch and open a PR against it.
Would that work for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3875300255
@adutra , I have created https://github.com/apache/polaris/pull/3719/changes on the feature branch. I built Iceberg locally to develop it. Once Iceberg nightly builds become available the test in that PR should pass. I think I have addressed all your review comments, but if I miss anything please let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2785812032
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -330,6 +357,80 @@ public Table registerTable(TableIdentifier identifier, String metadataFileLocati
return new BaseTable(ops, fullTableName(name(), identifier), metricsReporter());
}
+ private Table overwriteRegisteredTable(
+ TableIdentifier identifier, String metadataFileLocation, String locationDir) {
+ /*
+ * High-level overview:
+ * - Resolve the authorized parent for the table so we know the storage context.
+ * - Validate and read the provided metadata file using a FileIO tied to that
+ * storage context.
+ * - Ensure the target entity exists and is the correct table-like subtype.
+ * - Replace the stored entity properties (including metadata-location) with
+ * values derived from the parsed TableMetadata and persist the change.
+ */
+
+ // Resolve the parent namespace path that was authorized for this catalog.
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getResolvedPath(identifier.namespace());
+ if (resolvedParent == null) {
+ throw new IllegalStateException(
+ String.format("Failed to fetch resolved parent for TableIdentifier '%s'", identifier));
+ }
+
+ // Validate the supplied metadata file location against the resolved storage.
+ validateLocationForTableLike(identifier, metadataFileLocation, resolvedParent);
+
+ // Configure FileIO for the resolved storage and read the metadata file.
+ FileIO fileIO =
+ loadFileIOForTableLike(
+ identifier,
+ Set.of(locationDir),
+ resolvedParent,
+ new HashMap<>(tableDefaultProperties),
+ Set.of(PolarisStorageActions.READ, PolarisStorageActions.LIST));
+
+ TableMetadata metadata = TableMetadataParser.read(fileIO, metadataFileLocation);
+ validateLocationForTableLike(identifier, metadata.location(), resolvedParent);
+ validateMetadataFileInTableDir(identifier, metadata);
+
+ // Find the passthrough-resolved entity so we can update the stored record.
+ PolarisResolvedPathWrapper resolvedPath =
+ resolvedEntityView.getPassthroughResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ANY_SUBTYPE);
+ if (resolvedPath == null || resolvedPath.getRawLeafEntity() == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Ensure the raw entity is an Iceberg table-like entity (not a view/generic table).
+ PolarisEntity rawEntity = resolvedPath.getRawLeafEntity();
+ if (rawEntity.getSubType() == PolarisEntitySubType.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("View with same name already exists: %s", identifier);
+ } else if (rawEntity.getSubType() == PolarisEntitySubType.GENERIC_TABLE) {
+ throw new AlreadyExistsException(
+ "Generic table with same name already exists: %s", identifier);
+ }
+
+ IcebergTableLikeEntity existingEntity = IcebergTableLikeEntity.of(rawEntity);
+ if (existingEntity == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Build updated entity from parsed metadata and persist the update.
+ Map<String, String> storedProperties = buildTableMetadataPropertiesMap(metadata);
+ IcebergTableLikeEntity updatedEntity =
+ new IcebergTableLikeEntity.Builder(existingEntity)
+ .setInternalProperties(storedProperties)
+ .setMetadataLocation(metadataFileLocation)
+ .build();
+
+ updateTableLike(identifier, updatedEntity);
Review Comment:
Yes, a new UUID will be created. Added to the test to verify this in https://github.com/apache/polaris/pull/3719
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773912000
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/RegisterTableRequestContext.java:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.catalog.iceberg;
+
+/**
+ * Thread-local context that carries additional parameters for a {@code RegisterTableRequest} which
+ * are not present on the generated Iceberg request type.
+ *
+ * <p>The current implementation only records the caller's intent to "overwrite" an existing table.
+ * Using a ThreadLocal keeps the flag local to the request lifecycle and avoids changing the
+ * generated DTOs. Callers must clear the flag (typically in a finally block) to avoid leaking state
+ * across requests.
+ */
+public class RegisterTableRequestContext {
+ private static final ThreadLocal<Boolean> SHOULD_OVERWRITE = ThreadLocal.withInitial(() -> false);
Review Comment:
ThreadLocals are not recommended with Quarkus because there is no guarantee that a request will be handled by a single thread for the whole request scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3863074132
> > I have an additional question. My understanding is that once your PR in the iceberg repository gets merged and released as part of the next Iceberg update. We would have to update https://github.com/apache/polaris/blob/main/gradle/libs.versions.toml#L23 to pick up that artifact. Correct?
>
> You are correct in saying that this feature will have to wait for Iceberg 1.11.
>
> But development can happen ahead of schedule! I just created the `feature/iceberg-1.11` branch.
>
> Once the overwrite parameter is merged in Iceberg you would be able to rebase your current work on top of that branch and open a PR against it.
>
> Would that work for you?
Yes, I will update my PR in the meantime. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773498080
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -330,6 +357,80 @@ public Table registerTable(TableIdentifier identifier, String metadataFileLocati
return new BaseTable(ops, fullTableName(name(), identifier), metricsReporter());
}
+ private Table overwriteRegisteredTable(
+ TableIdentifier identifier, String metadataFileLocation, String locationDir) {
+ /*
+ * High-level overview:
+ * - Resolve the authorized parent for the table so we know the storage context.
+ * - Validate and read the provided metadata file using a FileIO tied to that
+ * storage context.
+ * - Ensure the target entity exists and is the correct table-like subtype.
+ * - Replace the stored entity properties (including metadata-location) with
+ * values derived from the parsed TableMetadata and persist the change.
+ */
+
+ // Resolve the parent namespace path that was authorized for this catalog.
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getResolvedPath(identifier.namespace());
+ if (resolvedParent == null) {
+ throw new IllegalStateException(
+ String.format("Failed to fetch resolved parent for TableIdentifier '%s'", identifier));
+ }
+
+ // Validate the supplied metadata file location against the resolved storage.
+ validateLocationForTableLike(identifier, metadataFileLocation, resolvedParent);
+
+ // Configure FileIO for the resolved storage and read the metadata file.
+ FileIO fileIO =
+ loadFileIOForTableLike(
+ identifier,
+ Set.of(locationDir),
+ resolvedParent,
+ new HashMap<>(tableDefaultProperties),
+ Set.of(PolarisStorageActions.READ, PolarisStorageActions.LIST));
+
+ TableMetadata metadata = TableMetadataParser.read(fileIO, metadataFileLocation);
+ validateLocationForTableLike(identifier, metadata.location(), resolvedParent);
+ validateMetadataFileInTableDir(identifier, metadata);
+
+ // Find the passthrough-resolved entity so we can update the stored record.
+ PolarisResolvedPathWrapper resolvedPath =
+ resolvedEntityView.getPassthroughResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ANY_SUBTYPE);
+ if (resolvedPath == null || resolvedPath.getRawLeafEntity() == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Ensure the raw entity is an Iceberg table-like entity (not a view/generic table).
+ PolarisEntity rawEntity = resolvedPath.getRawLeafEntity();
+ if (rawEntity.getSubType() == PolarisEntitySubType.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("View with same name already exists: %s", identifier);
+ } else if (rawEntity.getSubType() == PolarisEntitySubType.GENERIC_TABLE) {
+ throw new AlreadyExistsException(
+ "Generic table with same name already exists: %s", identifier);
+ }
+
+ IcebergTableLikeEntity existingEntity = IcebergTableLikeEntity.of(rawEntity);
+ if (existingEntity == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Build updated entity from parsed metadata and persist the update.
+ Map<String, String> storedProperties = buildTableMetadataPropertiesMap(metadata);
+ IcebergTableLikeEntity updatedEntity =
+ new IcebergTableLikeEntity.Builder(existingEntity)
+ .setInternalProperties(storedProperties)
+ .setMetadataLocation(metadataFileLocation)
+ .build();
+
+ updateTableLike(identifier, updatedEntity);
Review Comment:
Are we keeping the old table UUID here, or switching to the new UUID? I think the expected semantics is to switch to the new one.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java:
##########
@@ -348,7 +349,21 @@ public LoadTableResponse registerTable(
request.validate();
TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
- Table table = catalog.registerTable(identifier, request.metadataLocation());
+ // Determine whether the client requested overwrite semantics. For catalogs that
Review Comment:
This class is copied from Iceberg, I'm not sure it's a good idea to modify it. I would move the Polaris-specific logic to `IcebergCatalog` or `IcebergCatalogHandler`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773463756
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
Review Comment:
Why? Implementation details of the authorizer shouldn't matter here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra closed pull request #3682: [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites
URL: https://github.com/apache/polaris/pull/3682
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2785812032
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -330,6 +357,80 @@ public Table registerTable(TableIdentifier identifier, String metadataFileLocati
return new BaseTable(ops, fullTableName(name(), identifier), metricsReporter());
}
+ private Table overwriteRegisteredTable(
+ TableIdentifier identifier, String metadataFileLocation, String locationDir) {
+ /*
+ * High-level overview:
+ * - Resolve the authorized parent for the table so we know the storage context.
+ * - Validate and read the provided metadata file using a FileIO tied to that
+ * storage context.
+ * - Ensure the target entity exists and is the correct table-like subtype.
+ * - Replace the stored entity properties (including metadata-location) with
+ * values derived from the parsed TableMetadata and persist the change.
+ */
+
+ // Resolve the parent namespace path that was authorized for this catalog.
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getResolvedPath(identifier.namespace());
+ if (resolvedParent == null) {
+ throw new IllegalStateException(
+ String.format("Failed to fetch resolved parent for TableIdentifier '%s'", identifier));
+ }
+
+ // Validate the supplied metadata file location against the resolved storage.
+ validateLocationForTableLike(identifier, metadataFileLocation, resolvedParent);
+
+ // Configure FileIO for the resolved storage and read the metadata file.
+ FileIO fileIO =
+ loadFileIOForTableLike(
+ identifier,
+ Set.of(locationDir),
+ resolvedParent,
+ new HashMap<>(tableDefaultProperties),
+ Set.of(PolarisStorageActions.READ, PolarisStorageActions.LIST));
+
+ TableMetadata metadata = TableMetadataParser.read(fileIO, metadataFileLocation);
+ validateLocationForTableLike(identifier, metadata.location(), resolvedParent);
+ validateMetadataFileInTableDir(identifier, metadata);
+
+ // Find the passthrough-resolved entity so we can update the stored record.
+ PolarisResolvedPathWrapper resolvedPath =
+ resolvedEntityView.getPassthroughResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ANY_SUBTYPE);
+ if (resolvedPath == null || resolvedPath.getRawLeafEntity() == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Ensure the raw entity is an Iceberg table-like entity (not a view/generic table).
+ PolarisEntity rawEntity = resolvedPath.getRawLeafEntity();
+ if (rawEntity.getSubType() == PolarisEntitySubType.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("View with same name already exists: %s", identifier);
+ } else if (rawEntity.getSubType() == PolarisEntitySubType.GENERIC_TABLE) {
+ throw new AlreadyExistsException(
+ "Generic table with same name already exists: %s", identifier);
+ }
+
+ IcebergTableLikeEntity existingEntity = IcebergTableLikeEntity.of(rawEntity);
+ if (existingEntity == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Build updated entity from parsed metadata and persist the update.
+ Map<String, String> storedProperties = buildTableMetadataPropertiesMap(metadata);
+ IcebergTableLikeEntity updatedEntity =
+ new IcebergTableLikeEntity.Builder(existingEntity)
+ .setInternalProperties(storedProperties)
+ .setMetadataLocation(metadataFileLocation)
+ .build();
+
+ updateTableLike(identifier, updatedEntity);
Review Comment:
Yes, a new UUID will be created. Added to the test to verify this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773504142
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
Review Comment:
This is odd, the exact authorizer being used shouldn't matter. The rest of this method feels too low-level for this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773469615
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
+ return;
+ }
+
+ PolarisResolvedPathWrapper target =
+ resolutionManifest.getResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE, true);
+ if (target == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ Set<Long> activatedEntityIds =
+ resolutionManifest.getAllActivatedCatalogRoleAndPrincipalRoles().stream()
+ .map(PolarisEntityCore::getId)
+ .collect(Collectors.toSet());
+ boolean hasNonDataWritePrivilege =
+ hasWritePropertiesPrivilegeExcludingWriteData(authorizerImpl, activatedEntityIds, target);
+ if (!hasNonDataWritePrivilege) {
+ throw new ForbiddenException(
+ "Principal '%s' with activated PrincipalRoles '%s' and activated grants via '%s' is not authorized for op %s",
+ polarisPrincipal.getName(),
+ polarisPrincipal.getRoles(),
+ resolutionManifest.getAllActivatedCatalogRoleAndPrincipalRoles().stream()
+ .map(PolarisEntityCore::getName)
+ .collect(Collectors.toSet()),
+ PolarisAuthorizableOperation.UPDATE_TABLE);
+ }
+ }
+
+ private boolean hasWritePropertiesPrivilegeExcludingWriteData(
+ PolarisAuthorizerImpl authorizerImpl,
+ Set<Long> activatedEntityIds,
+ PolarisResolvedPathWrapper target) {
+ for (ResolvedPolarisEntity resolvedSecurableEntity : target.getResolvedFullPath()) {
+ for (PolarisGrantRecord grantRecord : resolvedSecurableEntity.getGrantRecordsAsSecurable()) {
+ if (activatedEntityIds.contains(grantRecord.getGranteeId())) {
+ PolarisPrivilege grantedPrivilege =
+ PolarisPrivilege.fromCode(grantRecord.getPrivilegeCode());
+ if (grantedPrivilege == PolarisPrivilege.TABLE_WRITE_DATA) {
+ continue;
+ }
+ if (authorizerImpl.matchesOrIsSubsumedBy(
+ PolarisPrivilege.TABLE_WRITE_PROPERTIES, grantedPrivilege)) {
+ return true;
+ }
+ }
+ }
+ }
+ return false;
Review Comment:
This feels very low-level and like it should be in the authorizer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3861887788
Thanks @adutra for your review.
I noticed that this this parameter is missing from the Iceberg artifact, so I implemented this PR in such a way that the polaris still supports it even if it was missing from Iceberg. But, I now realize that is the wrong approach to take. Thanks for opening the PR in the Iceberg repository adding support for overwrite. Once your PR get's merged, I will update this PR based on your review comments.
I have an additional question. My understanding is that once your PR in the iceberg repository gets merged and released as part of the next Iceberg update. We would have to update https://github.com/apache/polaris/blob/main/gradle/libs.versions.toml#L23 to pick up that artifact. Correct?
So we can only merge this feature in polaris after the next Iceberg release and after polaris picks up the updated artifact.. Is my understanding correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2785819656
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
Review Comment:
Fixed in the latest PR https://github.com/apache/polaris/pull/3719
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/RegisterTableRequestContext.java:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.catalog.iceberg;
+
+/**
+ * Thread-local context that carries additional parameters for a {@code RegisterTableRequest} which
+ * are not present on the generated Iceberg request type.
+ *
+ * <p>The current implementation only records the caller's intent to "overwrite" an existing table.
+ * Using a ThreadLocal keeps the flag local to the request lifecycle and avoids changing the
+ * generated DTOs. Callers must clear the flag (typically in a finally block) to avoid leaking state
+ * across requests.
+ */
+public class RegisterTableRequestContext {
+ private static final ThreadLocal<Boolean> SHOULD_OVERWRITE = ThreadLocal.withInitial(() -> false);
Review Comment:
Removed this entirely in https://github.com/apache/polaris/pull/3719
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3859141135
Hi @sririshindra thanks for opening this PR!
Unfortunately I don't think this is the right way to bring support for overwrite in register table requests. In fact, support for this parameter seems to be missing from the Iceberg artifact. So, I went ahead and created a PR to bring support for it in Iceberg: https://github.com/apache/iceberg/pull/15248.
If/when that PR is accepted, then we will be able to adapt Polaris properly to the new parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2773469615
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
+ return;
+ }
+
+ PolarisResolvedPathWrapper target =
+ resolutionManifest.getResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE, true);
+ if (target == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ Set<Long> activatedEntityIds =
+ resolutionManifest.getAllActivatedCatalogRoleAndPrincipalRoles().stream()
+ .map(PolarisEntityCore::getId)
+ .collect(Collectors.toSet());
+ boolean hasNonDataWritePrivilege =
+ hasWritePropertiesPrivilegeExcludingWriteData(authorizerImpl, activatedEntityIds, target);
+ if (!hasNonDataWritePrivilege) {
+ throw new ForbiddenException(
+ "Principal '%s' with activated PrincipalRoles '%s' and activated grants via '%s' is not authorized for op %s",
+ polarisPrincipal.getName(),
+ polarisPrincipal.getRoles(),
+ resolutionManifest.getAllActivatedCatalogRoleAndPrincipalRoles().stream()
+ .map(PolarisEntityCore::getName)
+ .collect(Collectors.toSet()),
+ PolarisAuthorizableOperation.UPDATE_TABLE);
+ }
+ }
+
+ private boolean hasWritePropertiesPrivilegeExcludingWriteData(
+ PolarisAuthorizerImpl authorizerImpl,
+ Set<Long> activatedEntityIds,
+ PolarisResolvedPathWrapper target) {
+ for (ResolvedPolarisEntity resolvedSecurableEntity : target.getResolvedFullPath()) {
+ for (PolarisGrantRecord grantRecord : resolvedSecurableEntity.getGrantRecordsAsSecurable()) {
+ if (activatedEntityIds.contains(grantRecord.getGranteeId())) {
+ PolarisPrivilege grantedPrivilege =
+ PolarisPrivilege.fromCode(grantRecord.getPrivilegeCode());
+ if (grantedPrivilege == PolarisPrivilege.TABLE_WRITE_DATA) {
+ continue;
+ }
+ if (authorizerImpl.matchesOrIsSubsumedBy(
+ PolarisPrivilege.TABLE_WRITE_PROPERTIES, grantedPrivilege)) {
+ return true;
+ }
+ }
+ }
+ }
+ return false;
Review Comment:
This feels very low-level and like it should be in the authorizer.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
Review Comment:
Why? Implementation details of the authorizer shouldn't matter here.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java:
##########
@@ -348,7 +349,21 @@ public LoadTableResponse registerTable(
request.validate();
TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
- Table table = catalog.registerTable(identifier, request.metadataLocation());
+ // Determine whether the client requested overwrite semantics. For catalogs that
Review Comment:
This class was copied from Iceberg. I'm not sure it's a good idea to modify it in a divergent way. I would place the Polaris-specific logic in `IcebergCatalogHandler` or `IcebergCatalog` directly.
##########
runtime/service/src/main/java/org/apache/polaris/service/config/Serializers.java:
##########
@@ -58,6 +61,7 @@ public static void registerSerializers(ObjectMapper mapper) {
GrantCatalogRoleRequest.class, new GrantCatalogRoleRequestDeserializer());
module.addDeserializer(AddGrantRequest.class, new AddGrantRequestDeserializer());
module.addDeserializer(RevokeGrantRequest.class, new RevokeGrantRequestDeserializer());
+ module.addDeserializer(RegisterTableRequest.class, new RegisterTableRequestDeserializer());
Review Comment:
This is going to conflict with `org.apache.iceberg.rest.RESTSerializers.RegisterTableRequestDeserializer`.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -330,6 +357,80 @@ public Table registerTable(TableIdentifier identifier, String metadataFileLocati
return new BaseTable(ops, fullTableName(name(), identifier), metricsReporter());
}
+ private Table overwriteRegisteredTable(
+ TableIdentifier identifier, String metadataFileLocation, String locationDir) {
+ /*
+ * High-level overview:
+ * - Resolve the authorized parent for the table so we know the storage context.
+ * - Validate and read the provided metadata file using a FileIO tied to that
+ * storage context.
+ * - Ensure the target entity exists and is the correct table-like subtype.
+ * - Replace the stored entity properties (including metadata-location) with
+ * values derived from the parsed TableMetadata and persist the change.
+ */
+
+ // Resolve the parent namespace path that was authorized for this catalog.
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getResolvedPath(identifier.namespace());
+ if (resolvedParent == null) {
+ throw new IllegalStateException(
+ String.format("Failed to fetch resolved parent for TableIdentifier '%s'", identifier));
+ }
+
+ // Validate the supplied metadata file location against the resolved storage.
+ validateLocationForTableLike(identifier, metadataFileLocation, resolvedParent);
+
+ // Configure FileIO for the resolved storage and read the metadata file.
+ FileIO fileIO =
+ loadFileIOForTableLike(
+ identifier,
+ Set.of(locationDir),
+ resolvedParent,
+ new HashMap<>(tableDefaultProperties),
+ Set.of(PolarisStorageActions.READ, PolarisStorageActions.LIST));
+
+ TableMetadata metadata = TableMetadataParser.read(fileIO, metadataFileLocation);
+ validateLocationForTableLike(identifier, metadata.location(), resolvedParent);
+ validateMetadataFileInTableDir(identifier, metadata);
+
+ // Find the passthrough-resolved entity so we can update the stored record.
+ PolarisResolvedPathWrapper resolvedPath =
+ resolvedEntityView.getPassthroughResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ANY_SUBTYPE);
+ if (resolvedPath == null || resolvedPath.getRawLeafEntity() == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Ensure the raw entity is an Iceberg table-like entity (not a view/generic table).
+ PolarisEntity rawEntity = resolvedPath.getRawLeafEntity();
+ if (rawEntity.getSubType() == PolarisEntitySubType.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("View with same name already exists: %s", identifier);
+ } else if (rawEntity.getSubType() == PolarisEntitySubType.GENERIC_TABLE) {
+ throw new AlreadyExistsException(
+ "Generic table with same name already exists: %s", identifier);
+ }
+
+ IcebergTableLikeEntity existingEntity = IcebergTableLikeEntity.of(rawEntity);
+ if (existingEntity == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Build updated entity from parsed metadata and persist the update.
+ Map<String, String> storedProperties = buildTableMetadataPropertiesMap(metadata);
+ IcebergTableLikeEntity updatedEntity =
+ new IcebergTableLikeEntity.Builder(existingEntity)
+ .setInternalProperties(storedProperties)
+ .setMetadataLocation(metadataFileLocation)
+ .build();
+
+ updateTableLike(identifier, updatedEntity);
Review Comment:
Are we keeping the old table UUID here, or switching to the new UUID? I think the expected semantics is to switch to the new one.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -330,6 +357,80 @@ public Table registerTable(TableIdentifier identifier, String metadataFileLocati
return new BaseTable(ops, fullTableName(name(), identifier), metricsReporter());
}
+ private Table overwriteRegisteredTable(
+ TableIdentifier identifier, String metadataFileLocation, String locationDir) {
+ /*
+ * High-level overview:
+ * - Resolve the authorized parent for the table so we know the storage context.
+ * - Validate and read the provided metadata file using a FileIO tied to that
+ * storage context.
+ * - Ensure the target entity exists and is the correct table-like subtype.
+ * - Replace the stored entity properties (including metadata-location) with
+ * values derived from the parsed TableMetadata and persist the change.
+ */
+
+ // Resolve the parent namespace path that was authorized for this catalog.
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getResolvedPath(identifier.namespace());
+ if (resolvedParent == null) {
+ throw new IllegalStateException(
+ String.format("Failed to fetch resolved parent for TableIdentifier '%s'", identifier));
+ }
+
+ // Validate the supplied metadata file location against the resolved storage.
+ validateLocationForTableLike(identifier, metadataFileLocation, resolvedParent);
+
+ // Configure FileIO for the resolved storage and read the metadata file.
+ FileIO fileIO =
+ loadFileIOForTableLike(
+ identifier,
+ Set.of(locationDir),
+ resolvedParent,
+ new HashMap<>(tableDefaultProperties),
+ Set.of(PolarisStorageActions.READ, PolarisStorageActions.LIST));
+
+ TableMetadata metadata = TableMetadataParser.read(fileIO, metadataFileLocation);
+ validateLocationForTableLike(identifier, metadata.location(), resolvedParent);
+ validateMetadataFileInTableDir(identifier, metadata);
+
+ // Find the passthrough-resolved entity so we can update the stored record.
+ PolarisResolvedPathWrapper resolvedPath =
+ resolvedEntityView.getPassthroughResolvedPath(
+ identifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ANY_SUBTYPE);
+ if (resolvedPath == null || resolvedPath.getRawLeafEntity() == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Ensure the raw entity is an Iceberg table-like entity (not a view/generic table).
+ PolarisEntity rawEntity = resolvedPath.getRawLeafEntity();
+ if (rawEntity.getSubType() == PolarisEntitySubType.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("View with same name already exists: %s", identifier);
+ } else if (rawEntity.getSubType() == PolarisEntitySubType.GENERIC_TABLE) {
+ throw new AlreadyExistsException(
+ "Generic table with same name already exists: %s", identifier);
+ }
+
+ IcebergTableLikeEntity existingEntity = IcebergTableLikeEntity.of(rawEntity);
+ if (existingEntity == null) {
+ throw new NoSuchTableException("Table does not exist: %s", identifier);
+ }
+
+ // Build updated entity from parsed metadata and persist the update.
+ Map<String, String> storedProperties = buildTableMetadataPropertiesMap(metadata);
+ IcebergTableLikeEntity updatedEntity =
+ new IcebergTableLikeEntity.Builder(existingEntity)
+ .setInternalProperties(storedProperties)
+ .setMetadataLocation(metadataFileLocation)
+ .build();
+
+ updateTableLike(identifier, updatedEntity);
Review Comment:
Is this switching the table UUID to the new UUID, or keeping the old UUID? From this ML discussions, I think it should be the new UUID:
https://lists.apache.org/thread/b5k7vdng904zr3n3q8wv83y8l30rnd4c
https://lists.apache.org/thread/k3595bttvohb6c3ms36o16gppdfllqmp
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java:
##########
@@ -348,7 +349,21 @@ public LoadTableResponse registerTable(
request.validate();
TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
- Table table = catalog.registerTable(identifier, request.metadataLocation());
+ // Determine whether the client requested overwrite semantics. For catalogs that
Review Comment:
This class is copied from Iceberg, I'm not sure it's a good idea to modify it. I would move the Polaris-specific logic to `IcebergCatalog` or `IcebergCatalogHandler`.
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -634,11 +640,110 @@ public LoadTableResponse createTableStaged(
* @return ETagged {@link LoadTableResponse} to uniquely identify the table metadata
*/
public LoadTableResponse registerTable(Namespace namespace, RegisterTableRequest request) {
- PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
- authorizeCreateTableLikeUnderNamespaceOperationOrThrow(
- op, TableIdentifier.of(namespace, request.name()));
+ TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
+ boolean overwrite = RegisterTableRequestContext.getOverwrite();
+
+ if (overwrite) {
+ // Resolve the namespace and table (optional) so we can distinguish overwrite from create.
+ resolutionManifest = newResolutionManifest();
+ resolutionManifest.addPath(
+ new ResolverPath(Arrays.asList(namespace.levels()), PolarisEntityType.NAMESPACE),
+ namespace);
+ ResolverPath tablePath =
+ new ResolverPath(
+ PolarisCatalogHelpers.tableIdentifierToList(identifier),
+ PolarisEntityType.TABLE_LIKE,
+ true /* optional */);
+ resolutionManifest.addPassthroughPath(tablePath, identifier);
+ resolutionManifest.resolveAll();
+
+ boolean tableExists =
+ resolutionManifest.getResolvedPath(
+ identifier,
+ PolarisEntityType.TABLE_LIKE,
+ PolarisEntitySubType.ICEBERG_TABLE,
+ true)
+ != null;
+ if (tableExists) {
+ authorizeUpdateTableOverwriteOrThrow(identifier);
+ LOGGER.debug(
+ "registerTable: overwrite=true, authorized for UPDATE_TABLE on existing table");
+ } else {
+ // Table doesn't exist, fall back to REGISTER_TABLE authorization
+ LOGGER.debug(
+ "registerTable: overwrite=true, table not found, falling back to REGISTER_TABLE");
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+ } else {
+ // Creating new table requires REGISTER_TABLE privilege
+ PolarisAuthorizableOperation op = PolarisAuthorizableOperation.REGISTER_TABLE;
+ authorizeCreateTableLikeUnderNamespaceOperationOrThrow(op, identifier);
+ }
+
+ try {
+ return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ } finally {
+ // Clean up context
+ RegisterTableRequestContext.clear();
+ }
+ }
+
+ private void authorizeUpdateTableOverwriteOrThrow(TableIdentifier identifier) {
+ authorizeBasicTableLikeOperationsOrThrow(
+ EnumSet.of(PolarisAuthorizableOperation.UPDATE_TABLE),
+ PolarisEntitySubType.ICEBERG_TABLE,
+ identifier);
- return catalogHandlerUtils.registerTable(baseCatalog, namespace, request);
+ if (!(authorizer instanceof PolarisAuthorizerImpl authorizerImpl)) {
Review Comment:
This is odd, the exact authorizer being used shouldn't matter. The rest of this method feels to low-level for this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on code in PR #3682:
URL: https://github.com/apache/polaris/pull/3682#discussion_r2785780192
##########
runtime/service/src/main/java/org/apache/polaris/service/config/Serializers.java:
##########
@@ -58,6 +61,7 @@ public static void registerSerializers(ObjectMapper mapper) {
GrantCatalogRoleRequest.class, new GrantCatalogRoleRequestDeserializer());
module.addDeserializer(AddGrantRequest.class, new AddGrantRequestDeserializer());
module.addDeserializer(RevokeGrantRequest.class, new RevokeGrantRequestDeserializer());
+ module.addDeserializer(RegisterTableRequest.class, new RegisterTableRequestDeserializer());
Review Comment:
Removed in https://github.com/apache/polaris/pull/3719
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java:
##########
@@ -348,7 +349,21 @@ public LoadTableResponse registerTable(
request.validate();
TableIdentifier identifier = TableIdentifier.of(namespace, request.name());
- Table table = catalog.registerTable(identifier, request.metadataLocation());
+ // Determine whether the client requested overwrite semantics. For catalogs that
Review Comment:
Removed in https://github.com/apache/polaris/pull/3719
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3873209435
The Iceberg PR has been merged FYI: https://github.com/apache/iceberg/pull/15248
We should be able to work on this feature against the `feature/iceberg-1.11` branch as soon as we have the next Iceberg nightly build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DRAFT] feat: add RegisterTable overwrite support; preserve correct auth for overwrites [polaris]
Posted by "sririshindra (via GitHub)" <gi...@apache.org>.
sririshindra commented on PR #3682:
URL: https://github.com/apache/polaris/pull/3682#issuecomment-3873911105
> The Iceberg PR has been merged FYI: [apache/iceberg#15248](https://github.com/apache/iceberg/pull/15248)
>
> We should be able to work on this feature against the `feature/iceberg-1.11` branch as soon as we have the next Iceberg nightly build.
Thanks @adutra I will update the Pr shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org