You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@polaris.apache.org by di...@apache.org on 2026/09/22 22:28:56 UTC
(polaris) branch main updated: Validate table-notification location before auto-creating parent namespaces (#5541)
This is an automated email from the ASF dual-hosted git repository.
dimas-b pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push:
new c179026373 Validate table-notification location before auto-creating parent namespaces (#5541)
c179026373 is described below
commit c179026373af02f3f89bd182e3e7f927407e0eac
Author: Prithvi S <pr...@gmail.com>
AuthorDate: Wed Sep 23 03:58:51 2026 +0530
Validate table-notification location before auto-creating parent namespaces (#5541)
CREATE/UPDATE table notifications auto-create any missing parent
namespaces via createNonExistingNamespaces before validating the
proposed metadata location. A notification carrying a location outside
the catalog's allowed locations therefore persisted the new namespaces
first and only then failed validation, leaving them orphaned. It also
let a principal holding only SEND_NOTIFICATIONS create namespaces as a
side effect of a request that ultimately fails.
Validate the location before creating namespaces. Because the location
check needs the storage configuration of the parent hierarchy - which
may not exist yet - resolve the deepest already-existing ancestor
(down to the catalog root) and validate against it, mirroring what the
VALIDATE notification path already does. Auto-created namespaces inherit
their storage configuration from that same ancestor, so the decision is
unchanged for valid notifications. The shared walk-up logic is extracted
into resolveDeepestExistingStorageEntity and reused by both paths, and
the now-unused two-argument validateLocationForTableLike overload is
removed.
Adds a regression test asserting that a notification with a disallowed
location creates no namespaces.
---
CHANGELOG.md | 3 +
.../catalog/iceberg/LocalIcebergCatalog.java | 84 +++++++++++-----------
.../iceberg/AbstractLocalIcebergCatalogTest.java | 45 ++++++++++++
3 files changed, 89 insertions(+), 43 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e280beb50b..bc17218795 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -158,6 +158,9 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
matching `--credential` unless `--print-credentials` is given; otherwise the command names
the realms that are missing credentials and exits without bootstrapping anything.
`--credentials-file` is unaffected.
+- Table notifications (`CREATE`/`UPDATE`) that reference a metadata location outside the catalog's
+ allowed locations are now rejected before any missing parent namespaces are auto-created, so a
+ rejected notification no longer leaves orphaned namespaces behind.
- GCS credential vending no longer fails with HTTP 500 when a table's location or `write.data.path`
/ `write.metadata.path` points at a bucket root without a trailing slash (e.g. `gs://bucket`).
Such a location parses to an empty path and previously triggered a `StringIndexOutOfBoundsException`
diff --git a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java b/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java
index 43ff1dcb44..090770053f 100644
--- a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java
+++ b/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java
@@ -1326,27 +1326,6 @@ public class LocalIcebergCatalog extends BaseMetastoreViewCatalog
resolvedStorageEntity.getRawFullPath());
}
- /**
- * Validates that the specified {@code location} is valid for whatever storage config is found for
- * this TableLike's parent hierarchy.
- */
- private void validateLocationForTableLike(TableIdentifier identifier, String location) {
- PolarisResolvedPathWrapper resolvedStorageEntity =
- resolvedEntityView.getResolvedPath(
- ResolvedPathKey.ofTableLike(identifier), PolarisEntitySubType.ANY_SUBTYPE);
- if (resolvedStorageEntity == null) {
- resolvedStorageEntity =
- resolvedEntityView.getResolvedPath(ResolvedPathKey.ofNamespace(identifier.namespace()));
- }
- if (resolvedStorageEntity == null) {
- resolvedStorageEntity =
- resolvedEntityView.getPassthroughResolvedPath(
- ResolvedPathKey.ofNamespace(identifier.namespace()));
- }
-
- validateLocationForTableLike(identifier, location, resolvedStorageEntity);
- }
-
/**
* Validates that the specified {@code location} is valid for whatever storage config is found for
* this TableLike's parent hierarchy.
@@ -3036,23 +3015,13 @@ public class LocalIcebergCatalog extends BaseMetastoreViewCatalog
// of the TableIdentifier, which may even be the base CatalogEntity if no parent namespaces
// actually exist yet. We can then extract the right StorageInfo entity via a normal call
// to findStorageInfoFromHierarchy.
- PolarisResolvedPathWrapper resolvedStorageEntity = null;
- Optional<PolarisEntity> storageInfoEntity = Optional.empty();
- for (int i = tableIdentifier.namespace().length(); i >= 0; i--) {
- Namespace nsLevel =
- Namespace.of(
- Arrays.stream(tableIdentifier.namespace().levels())
- .limit(i)
- .toArray(String[]::new));
- resolvedStorageEntity =
- resolvedEntityView.getResolvedPath(ResolvedPathKey.ofNamespace(nsLevel));
- if (resolvedStorageEntity != null) {
- storageInfoEntity =
- PolarisStorageConfigurationInfo.findEntityWithStorageConfigFromHierarchy(
+ PolarisResolvedPathWrapper resolvedStorageEntity =
+ resolveDeepestExistingStorageEntity(tableIdentifier);
+ Optional<PolarisEntity> storageInfoEntity =
+ resolvedStorageEntity == null
+ ? Optional.empty()
+ : PolarisStorageConfigurationInfo.findEntityWithStorageConfigFromHierarchy(
resolvedStorageEntity);
- break;
- }
- }
if (resolvedStorageEntity == null || storageInfoEntity.isEmpty()) {
throw new BadRequestException(
@@ -3081,10 +3050,6 @@ public class LocalIcebergCatalog extends BaseMetastoreViewCatalog
|| notificationType == NotificationType.UPDATE) {
Namespace ns = tableIdentifier.namespace();
- createNonExistingNamespaces(ns);
-
- PolarisResolvedPathWrapper resolvedParent =
- resolvedEntityView.getPassthroughResolvedPath(ResolvedPathKey.ofNamespace(ns));
IcebergTableLikeEntity entity =
IcebergTableLikeEntity.of(
@@ -3119,8 +3084,19 @@ public class LocalIcebergCatalog extends BaseMetastoreViewCatalog
.setLastNotificationTimestamp(request.getPayload().getTimestamp())
.build();
}
- // first validate we can read the metadata file
- validateLocationForTableLike(tableIdentifier, newLocation);
+ // Validate the proposed metadata location against the storage configuration of the deepest
+ // already-existing ancestor BEFORE auto-creating any parent namespaces. Otherwise a
+ // notification carrying a disallowed location would fail validation only after the namespaces
+ // were persisted, leaving them orphaned. Auto-created namespaces inherit their storage
+ // configuration from this same ancestor, so validating here is equivalent to validating after
+ // creation.
+ validateLocationForTableLike(
+ tableIdentifier, newLocation, resolveDeepestExistingStorageEntity(tableIdentifier));
+
+ createNonExistingNamespaces(ns);
+
+ PolarisResolvedPathWrapper resolvedParent =
+ resolvedEntityView.getPassthroughResolvedPath(ResolvedPathKey.ofNamespace(ns));
String locationDir = newLocation.substring(0, newLocation.lastIndexOf("/"));
@@ -3164,6 +3140,28 @@ public class LocalIcebergCatalog extends BaseMetastoreViewCatalog
return true;
}
+ /**
+ * Walks up the namespace hierarchy of {@code identifier} (from the deepest level up to the
+ * catalog root) and returns the resolved path of the nearest ancestor that already exists. Since
+ * auto-created namespaces inherit their storage configuration from this ancestor, callers can use
+ * it to validate a proposed location before any namespaces are created. Returns {@code null} if
+ * no ancestor is resolved.
+ */
+ private PolarisResolvedPathWrapper resolveDeepestExistingStorageEntity(
+ TableIdentifier identifier) {
+ for (int i = identifier.namespace().length(); i >= 0; i--) {
+ Namespace nsLevel =
+ Namespace.of(
+ Arrays.stream(identifier.namespace().levels()).limit(i).toArray(String[]::new));
+ PolarisResolvedPathWrapper resolvedStorageEntity =
+ resolvedEntityView.getResolvedPath(ResolvedPathKey.ofNamespace(nsLevel));
+ if (resolvedStorageEntity != null) {
+ return resolvedStorageEntity;
+ }
+ }
+ return null;
+ }
+
private void createNonExistingNamespaces(Namespace namespace) {
// Pre-create namespaces if they don't exist
for (int i = 1; i <= namespace.length(); i++) {
diff --git a/runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogTest.java b/runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogTest.java
index b16733fc30..a8acafea35 100644
--- a/runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogTest.java
+++ b/runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogTest.java
@@ -1254,6 +1254,51 @@ public abstract class AbstractLocalIcebergCatalogTest extends CatalogTests<Local
.hasMessageContaining("Invalid location");
}
+ @Test
+ public void testNotificationInDisallowedLocationCreatesNoNamespaces() {
+ Assumptions.assumeTrue(
+ requiresNamespaceCreate(),
+ "Only applicable if namespaces must be created before adding children");
+ Assumptions.assumeTrue(
+ supportsNestedNamespaces(), "Only applicable if nested namespaces are supported");
+ Assumptions.assumeTrue(
+ supportsNotifications(), "Only applicable if notifications are supported");
+
+ // A notification whose metadata location is outside the catalog's allowed locations must be
+ // rejected before any parent namespaces are auto-created, so it cannot leave orphaned
+ // namespaces behind.
+ final String tableLocation = "s3://forbidden-table-location/table/";
+ final String tableMetadataLocation = tableLocation + "metadata/v1.metadata.json";
+ LocalIcebergCatalog catalog = catalog();
+
+ Namespace namespace = Namespace.of("parent", "child1");
+ TableIdentifier table = TableIdentifier.of(namespace, "table");
+
+ NotificationRequest request = new NotificationRequest();
+ request.setNotificationType(NotificationType.UPDATE);
+ TableUpdateNotification update = new TableUpdateNotification();
+ update.setMetadataLocation(tableMetadataLocation);
+ update.setTableName(table.name());
+ update.setTableUuid(UUID.randomUUID().toString());
+ update.setTimestamp(230950845L);
+ request.setPayload(update);
+
+ fileIO.addFile(
+ tableMetadataLocation,
+ TableMetadataParser.toJson(createSampleTableMetadata(tableLocation)).getBytes(UTF_8));
+
+ Assertions.assertThatThrownBy(() -> catalog.sendNotification(table, request))
+ .isInstanceOf(ForbiddenException.class)
+ .hasMessageContaining("Invalid location");
+
+ Assertions.assertThat(catalog.namespaceExists(namespace))
+ .as("Child namespace must not be created when the notification location is disallowed")
+ .isFalse();
+ Assertions.assertThat(catalog.namespaceExists(Namespace.of("parent")))
+ .as("Parent namespace must not be created when the notification location is disallowed")
+ .isFalse();
+ }
+
@Test
public void testCreateNotificationCreateTableInExternalLocation() {
Assumptions.assumeTrue(