You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "henrib (via GitHub)" <gi...@apache.org> on 2026/09/22 18:18:24 UTC
[PR] HIVE-30004: Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
henrib opened a new pull request, #6812:
URL: https://github.com/apache/hive/pull/6812
## Summary
- Reject REGISTER_TABLE requests whose metadata location or embedded table location is not authorized or contained within the namespace.
- Authorize DROP_TABLE purge against the table's location, and fence purge deletions to the table's own location as defense in depth.
## Test plan
- [x] `mvn -pl standalone-metastore/metastore-rest-catalog,iceberg/iceberg-catalog -am test` — 885 tests in `metastore-rest-catalog` (0 failures/errors, 45 skipped Docker-dependent), 142 tests in `iceberg-catalog` (0 failures/errors, 16 skipped)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-5879541570
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4124466509
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -241,6 +241,10 @@ public String name() {
return name;
}
+ public FileIO io() {
Review Comment:
Removed — `io()` is gone entirely now that `validateRegisterTable` no longer needs it (see below).
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
Review Comment:
Downgraded to DEBUG.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ @Override
+ public Map<String, String> properties() {
+ return delegate.properties();
+ }
+
+ @Override
+ public void initialize(Map<String, String> properties) {
+ delegate.initialize(properties);
+ }
+
+ @Override
+ public void close() {
+ delegate.close();
+ }
+
+ private static boolean isContained(String root, String candidate) {
+ return candidate.equals(root) || candidate.startsWith(root.endsWith("/") ? root : root + "/");
+ }
+
+ private static String normalize(String location) {
+ return new Path(location).toUri().normalize().toString();
+ }
Review Comment:
Extracted to `FileUtils.isPathWithinSubtree` in `metastore-common`, shared by both.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java:
##########
@@ -320,6 +325,10 @@ private LoadTableResponse loadTable(Map<String, String> vars) {
private LoadTableResponse registerTable(Map<String, String> vars, Object body) {
Namespace namespace = namespaceFromPathVars(vars);
RegisterTableRequest request = castRequest(RegisterTableRequest.class, body);
+ request.validate();
+ Map<String, String> namespaceMetadata = asNamespaceCatalog.loadNamespaceMetadata(namespace);
+ FileIO io = ((HiveCatalog) catalog).io();
+ icebergAuthorizer.validateRegisterTable(catalogName, namespace, namespaceMetadata, request, io);
return castResponse(LoadTableResponse.class, CatalogHandlers.registerTable(catalog, namespace, request));
Review Comment:
No longer casts to `HiveCatalog` at all — the `io`-based check it needed is removed.
##########
standalone-metastore/metastore-rest-catalog/src/test/java/org/apache/iceberg/rest/BaseRESTCatalogTests.java:
##########
@@ -277,4 +284,56 @@ void testStageCreateTableWithDeniedLocation() {
Assertions.assertThrows(ForbiddenException.class, builder::createTransaction);
Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
}
+
+ private static String writeMetadataFile(String directory, String tableLocation) throws IOException {
+ var metadataLocation = directory + "/v1.metadata.json";
+ Files.deleteIfExists(java.nio.file.Path.of(metadataLocation));
+ var io = new HadoopFileIO(new Configuration(false));
+ var metadata = TableMetadata.newTableMetadata(new Schema(), PartitionSpec.unpartitioned(), tableLocation,
+ Collections.emptyMap());
+ TableMetadataParser.write(metadata, io.newOutputFile(metadataLocation));
+ return metadataLocation;
+ }
Review Comment:
Leaving as-is: every location in this suite is rooted under the local test warehouse dir, so this isn't reachable in practice.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
Review Comment:
Agreed, keeping the check (per HIVE-28804).
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java:
##########
@@ -320,6 +325,10 @@ private LoadTableResponse loadTable(Map<String, String> vars) {
private LoadTableResponse registerTable(Map<String, String> vars, Object body) {
Namespace namespace = namespaceFromPathVars(vars);
RegisterTableRequest request = castRequest(RegisterTableRequest.class, body);
+ request.validate();
+ Map<String, String> namespaceMetadata = asNamespaceCatalog.loadNamespaceMetadata(namespace);
+ FileIO io = ((HiveCatalog) catalog).io();
Review Comment:
Agreed — removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin merged PR #6812:
URL: https://github.com/apache/hive/pull/6812
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-5991323856
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-6040761877
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-5962132697
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [6 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4169752235
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.metastore.utils.FileUtils;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final Path location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = new Path(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ var candidate = new Path(path);
+ if (!FileUtils.isPathWithinSubtree(qualify(candidate, location), qualify(location, candidate))) {
+ LOG.debug("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ /**
+ * A path with no scheme refers to the same filesystem location as an otherwise-identical path
+ * that does carry one, whichever side that happens to be: manifests can hold data file paths
+ * written without a scheme, and a table's own location can equally be scheme-less (e.g. when a
+ * caller supplies one explicitly). {@link org.apache.hadoop.fs.Path#equals} is scheme-sensitive,
+ * so without this a schemeless path would never match its scheme-qualified counterpart in
+ * {@link FileUtils#isPathWithinSubtree} even when it is nested directly under it. When both (or
+ * neither) sides carry a scheme, {@code path} is returned unchanged, so a genuine mismatch (e.g.
+ * a different scheme or authority) is still correctly treated as outside the table location.
+ */
+ private static Path qualify(Path path, Path other) {
+ var pathUri = path.toUri();
+ var otherUri = other.toUri();
+ if (pathUri.getScheme() == null && otherUri.getScheme() != null) {
+ return new Path(otherUri.getScheme(), otherUri.getAuthority(), pathUri.getPath());
Review Comment:
Qualifying a scheme-less path against whatever scheme the other side happened to carry wasn't quite right — a scheme-less path refers to a location on the cluster's default filesystem (`fs.defaultFS`), not to the comparison target's filesystem. `ScopedDeleteFileIO` now takes the Hadoop `Configuration` and qualifies both the candidate and the table location via `FileSystem.getDefaultUri(conf)` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4185817870
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
Review Comment:
I am checking whether DFS_URI should be validated as an input, an output, or both. I hit an issue where Iceberg REST -> HDFS access doesn't work. I'm checking why. I guess this is not a problem with HIVE-30002 but an issue on the master branch.
```
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] fs.FileSystem: Failed to initialize filesystem hdfs://zookage/apps/hive-warehouse/test/metadata/00000-b8ac910f-489a-438d-8c67-67c748a2123e.metadata.json: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] util.Tasks: Retrying task after failure: sleepTimeMs=108 java.net.UnknownHostException: zookage
hive-metastore-server-0: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:479)
hive-metastore-server-0: at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:134)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083740582
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
Review Comment:
I guess this is validated on CREATE TABLE.
https://github.com/apache/hive/blob/5ffc2f9991deea506931de3e1f886bdf50a37279/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/CreateTableEvent.java#L69-L74
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083904705
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
Review Comment:
I guess this is unnecessary, since a user is typically authorized per table, not per directory.
When a user has INSERT permission on table X, they can add a new data file to the location. Similarly, with DROP permission, they can delete files in the location.
This assumption is compromised when a compromised location belongs to a table. I think CREATE TABLE or ALTER TABLE properly checks DFS_URI, so DROP TABLE can assume the location is valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083904705
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
Review Comment:
~~I guess this is unnecessary, since a user is typically authorized per table, not per directory.
When a user has INSERT permission on table X, they can add a new data file to the location. Similarly, with DROP permission, they can delete files in the location.
This assumption is compromised when a compromised location belongs to a table. I think CREATE TABLE or ALTER TABLE properly checks DFS_URI, so DROP TABLE can assume the location is valid.~~
I changed my mind. Authorizing storage access could be a good idea.
https://issues.apache.org/jira/browse/HIVE-28804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-6073491218
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30004: Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083740582
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
Review Comment:
I guess this is validated on CREATE TABLE.
https://github.com/apache/hive/blob/5ffc2f9991deea506931de3e1f886bdf50a37279/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/CreateTableEvent.java#L69-L74
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4124468580
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
Review Comment:
Downgraded to DEBUG.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ @Override
+ public Map<String, String> properties() {
+ return delegate.properties();
+ }
+
+ @Override
+ public void initialize(Map<String, String> properties) {
+ delegate.initialize(properties);
+ }
+
+ @Override
+ public void close() {
+ delegate.close();
+ }
+
+ private static boolean isContained(String root, String candidate) {
+ return candidate.equals(root) || candidate.startsWith(root.endsWith("/") ? root : root + "/");
Review Comment:
Extracted to `FileUtils.isPathWithinSubtree` in `metastore-common`, shared by both.
##########
standalone-metastore/metastore-rest-catalog/src/test/java/org/apache/iceberg/rest/BaseRESTCatalogTests.java:
##########
@@ -277,4 +284,56 @@ void testStageCreateTableWithDeniedLocation() {
Assertions.assertThrows(ForbiddenException.class, builder::createTransaction);
Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
}
+
+ private static String writeMetadataFile(String directory, String tableLocation) throws IOException {
+ var metadataLocation = directory + "/v1.metadata.json";
+ Files.deleteIfExists(java.nio.file.Path.of(metadataLocation));
+ var io = new HadoopFileIO(new Configuration(false));
+ var metadata = TableMetadata.newTableMetadata(new Schema(), PartitionSpec.unpartitioned(), tableLocation,
+ Collections.emptyMap());
+ TableMetadataParser.write(metadata, io.newOutputFile(metadataLocation));
+ return metadataLocation;
+ }
+
+ @Test
+ void testRegisterTableWithDeniedLocation() {
+ var tableIdentifier = TableIdentifier.of("default", "register-table-denied");
+ var metadataLocation = MockHiveAuthorizer.DENIED_PREFIX + "/register-table-denied/v1.metadata.json";
+ Assertions.assertThrows(ForbiddenException.class, () -> catalog.registerTable(tableIdentifier, metadataLocation));
+ Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
+ }
+
+ @Test
+ void testRegisterTableWithDeniedEmbeddedLocation() throws IOException {
+ var tableIdentifier = TableIdentifier.of("default", "register-table-embedded-denied");
+ var tableLocation = MockHiveAuthorizer.DENIED_PREFIX + "/register-table-embedded-denied";
+ var metadataLocation = writeMetadataFile(
+ MockHiveAuthorizer.ALLOWED_PREFIX + "/register-table-embedded-denied", tableLocation);
+ Assertions.assertThrows(ForbiddenException.class, () -> catalog.registerTable(tableIdentifier, metadataLocation));
+ Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
+ }
+
+ @Test
+ void testDropTablePurgeDoesNotDeleteFilesOutsideTableLocation() throws IOException {
+ var victimDirectory = java.nio.file.Path.of(MockHiveAuthorizer.ALLOWED_PREFIX, "structural-fence-victim");
+ Files.createDirectories(victimDirectory);
+ var victimFile = victimDirectory.resolve("victim-data.txt");
+ Files.writeString(victimFile, "victim data");
+
+ var tableIdentifier = TableIdentifier.of("default", "structural-fence-attacker");
+ var tableLocation = MockHiveAuthorizer.ALLOWED_PREFIX + "/structural-fence-attacker";
+ Table table = catalog.buildTable(tableIdentifier, new Schema()).withLocation(tableLocation).create();
+
+ DataFile dataFile = DataFiles.builder(table.spec())
+ .withPath(victimFile.toUri().toString())
+ .withFormat(FileFormat.PARQUET)
+ .withFileSizeInBytes(Files.size(victimFile))
+ .withRecordCount(1)
+ .build();
+ table.newAppend().appendFile(dataFile).commit();
Review Comment:
Done.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges drop-table-purge", e);
+ }
+ }
+
+ private void checkLocationAuthorized(String catalogName, String databaseName,
+ Map<String, String> namespaceMetadata, String location, String commandString) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, falling back to namespace containment for {}",
+ catalogName, location);
+ checkContainedInNamespace(databaseName, namespaceMetadata, location);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString(commandString);
+ try {
+ authorizer.checkPrivileges(HiveOperationType.CREATETABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges for " + commandString, e);
+ }
+ }
+
+ private void checkContainedInNamespace(String databaseName, Map<String, String> namespaceMetadata,
+ String location) {
+ var externalRoot = namespaceMetadata.get("location");
+ if (externalRoot != null && isContained(externalRoot, location)) {
+ return;
+ }
+ if (isContained(managedNamespaceLocation(databaseName), location)) {
+ return;
+ }
+ throw new ForbiddenException(
+ "Location %s is not authorized and is not contained in namespace %s", location, databaseName);
+ }
+
+ private String managedNamespaceLocation(String databaseName) {
+ var warehouseLocation = conf.get(HiveConf.ConfVars.METASTORE_WAREHOUSE.varname);
+ Preconditions.checkNotNull(warehouseLocation, "Warehouse location is not set: hive.metastore.warehouse.dir=null");
+ if (warehouseLocation.endsWith("/")) {
+ warehouseLocation = warehouseLocation.substring(0, warehouseLocation.length() - 1);
+ }
+ return String.format("%s/%s.db", warehouseLocation, databaseName);
Review Comment:
Done — dropped the hand-rolled fallback, using the namespace metadata's own location.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges drop-table-purge", e);
+ }
+ }
+
+ private void checkLocationAuthorized(String catalogName, String databaseName,
+ Map<String, String> namespaceMetadata, String location, String commandString) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, falling back to namespace containment for {}",
+ catalogName, location);
+ checkContainedInNamespace(databaseName, namespaceMetadata, location);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString(commandString);
+ try {
+ authorizer.checkPrivileges(HiveOperationType.CREATETABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges for " + commandString, e);
+ }
+ }
+
+ private void checkContainedInNamespace(String databaseName, Map<String, String> namespaceMetadata,
+ String location) {
+ var externalRoot = namespaceMetadata.get("location");
+ if (externalRoot != null && isContained(externalRoot, location)) {
+ return;
+ }
+ if (isContained(managedNamespaceLocation(databaseName), location)) {
+ return;
+ }
+ throw new ForbiddenException(
+ "Location %s is not authorized and is not contained in namespace %s", location, databaseName);
+ }
+
+ private String managedNamespaceLocation(String databaseName) {
+ var warehouseLocation = conf.get(HiveConf.ConfVars.METASTORE_WAREHOUSE.varname);
+ Preconditions.checkNotNull(warehouseLocation, "Warehouse location is not set: hive.metastore.warehouse.dir=null");
+ if (warehouseLocation.endsWith("/")) {
+ warehouseLocation = warehouseLocation.substring(0, warehouseLocation.length() - 1);
+ }
+ return String.format("%s/%s.db", warehouseLocation, databaseName);
+ }
+
+ /**
+ * Checks whether {@code candidate} resolves under {@code root}. Both are resolved via {@link Path#toUri()} and
+ * {@link java.net.URI#normalize()}, which -- unlike {@link Path}'s own normalization -- actually collapses
+ * {@code .}/{@code ..} segments; a plain string-prefix comparison on unnormalized paths would let a location
+ * like {@code root/../../elsewhere} pass a naive check while actually resolving outside {@code root}.
+ */
+ private static boolean isContained(String root, String candidate) {
Review Comment:
Done, copied into `metastore-common` since this module doesn't depend on `hive-common`.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
Review Comment:
Done — removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4113885372
##########
standalone-metastore/metastore-rest-catalog/src/test/java/org/apache/iceberg/rest/BaseRESTCatalogTests.java:
##########
@@ -277,4 +284,56 @@ void testStageCreateTableWithDeniedLocation() {
Assertions.assertThrows(ForbiddenException.class, builder::createTransaction);
Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
}
+
+ private static String writeMetadataFile(String directory, String tableLocation) throws IOException {
+ var metadataLocation = directory + "/v1.metadata.json";
+ Files.deleteIfExists(java.nio.file.Path.of(metadataLocation));
+ var io = new HadoopFileIO(new Configuration(false));
+ var metadata = TableMetadata.newTableMetadata(new Schema(), PartitionSpec.unpartitioned(), tableLocation,
+ Collections.emptyMap());
+ TableMetadataParser.write(metadata, io.newOutputFile(metadataLocation));
+ return metadataLocation;
+ }
+
+ @Test
+ void testRegisterTableWithDeniedLocation() {
+ var tableIdentifier = TableIdentifier.of("default", "register-table-denied");
+ var metadataLocation = MockHiveAuthorizer.DENIED_PREFIX + "/register-table-denied/v1.metadata.json";
+ Assertions.assertThrows(ForbiddenException.class, () -> catalog.registerTable(tableIdentifier, metadataLocation));
+ Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
+ }
+
+ @Test
+ void testRegisterTableWithDeniedEmbeddedLocation() throws IOException {
+ var tableIdentifier = TableIdentifier.of("default", "register-table-embedded-denied");
+ var tableLocation = MockHiveAuthorizer.DENIED_PREFIX + "/register-table-embedded-denied";
+ var metadataLocation = writeMetadataFile(
+ MockHiveAuthorizer.ALLOWED_PREFIX + "/register-table-embedded-denied", tableLocation);
+ Assertions.assertThrows(ForbiddenException.class, () -> catalog.registerTable(tableIdentifier, metadataLocation));
+ Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
+ }
+
+ @Test
+ void testDropTablePurgeDoesNotDeleteFilesOutsideTableLocation() throws IOException {
+ var victimDirectory = java.nio.file.Path.of(MockHiveAuthorizer.ALLOWED_PREFIX, "structural-fence-victim");
+ Files.createDirectories(victimDirectory);
+ var victimFile = victimDirectory.resolve("victim-data.txt");
+ Files.writeString(victimFile, "victim data");
+
+ var tableIdentifier = TableIdentifier.of("default", "structural-fence-attacker");
+ var tableLocation = MockHiveAuthorizer.ALLOWED_PREFIX + "/structural-fence-attacker";
+ Table table = catalog.buildTable(tableIdentifier, new Schema()).withLocation(tableLocation).create();
+
+ DataFile dataFile = DataFiles.builder(table.spec())
+ .withPath(victimFile.toUri().toString())
+ .withFormat(FileFormat.PARQUET)
+ .withFileSizeInBytes(Files.size(victimFile))
+ .withRecordCount(1)
+ .build();
+ table.newAppend().appendFile(dataFile).commit();
Review Comment:
We may add one more data file under `MockHiveAuthorizer.ALLOWED_PREFIX + "/structural-fence-attacker/data"` and check whether the file is removed. It will explain how the file io works better.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges drop-table-purge", e);
+ }
+ }
+
+ private void checkLocationAuthorized(String catalogName, String databaseName,
+ Map<String, String> namespaceMetadata, String location, String commandString) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, falling back to namespace containment for {}",
+ catalogName, location);
+ checkContainedInNamespace(databaseName, namespaceMetadata, location);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString(commandString);
+ try {
+ authorizer.checkPrivileges(HiveOperationType.CREATETABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges for " + commandString, e);
+ }
+ }
+
+ private void checkContainedInNamespace(String databaseName, Map<String, String> namespaceMetadata,
+ String location) {
+ var externalRoot = namespaceMetadata.get("location");
+ if (externalRoot != null && isContained(externalRoot, location)) {
+ return;
+ }
+ if (isContained(managedNamespaceLocation(databaseName), location)) {
+ return;
+ }
+ throw new ForbiddenException(
+ "Location %s is not authorized and is not contained in namespace %s", location, databaseName);
+ }
+
+ private String managedNamespaceLocation(String databaseName) {
+ var warehouseLocation = conf.get(HiveConf.ConfVars.METASTORE_WAREHOUSE.varname);
+ Preconditions.checkNotNull(warehouseLocation, "Warehouse location is not set: hive.metastore.warehouse.dir=null");
+ if (warehouseLocation.endsWith("/")) {
+ warehouseLocation = warehouseLocation.substring(0, warehouseLocation.length() - 1);
+ }
+ return String.format("%s/%s.db", warehouseLocation, databaseName);
Review Comment:
Let's use `asNamespaceCatalog.loadNamespaceMetadata(namespace).get("location")` that abstracts the naming convention. Some conventions are exceptions; for example, the `default.test` table is located not in `/path/to/warehouse/default.db/test` but in `/path/to/warehouse/test`.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -271,7 +275,7 @@ public boolean dropTable(TableIdentifier identifier, boolean purge) {
});
if (purge && lastMetadata != null) {
- CatalogUtil.dropTableData(ops.io(), lastMetadata);
+ CatalogUtil.dropTableData(new ScopedDeleteFileIO(ops.io(), lastMetadata.location()), lastMetadata);
Review Comment:
This change disallows a user from purging metadata or data files outside the location, even if they have storage access. I think it is OK as the default value.
For future reference, [Polaris has some defensive configurations](https://polaris.apache.org/releases/1.7.0/configuration/configuration-reference/).
- DROP_WITH_PURGE_ENABLED: DROP with PURGE is disabled by default
- ALLOW_EXTERNAL_METADATA_FILE_LOCATION: Metadata files outside the location are not allowed by default
- ALLOW_UNSTRUCTURED_TABLE_LOCATION: The directory structure is enforced by default
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -241,6 +241,10 @@ public String name() {
return name;
}
+ public FileIO io() {
Review Comment:
And, maybe the name should be `ioForPurge` or else so that we don't mistakenly use it to scan files.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ @Override
+ public Map<String, String> properties() {
+ return delegate.properties();
+ }
+
+ @Override
+ public void initialize(Map<String, String> properties) {
+ delegate.initialize(properties);
+ }
+
+ @Override
+ public void close() {
+ delegate.close();
+ }
+
+ private static boolean isContained(String root, String candidate) {
+ return candidate.equals(root) || candidate.startsWith(root.endsWith("/") ? root : root + "/");
Review Comment:
I prefer to use `FileUtils.isPathWithinSubtree`.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
Review Comment:
```suggestion
```
We can remove this one since it falls within the scope of the CREATE TABLE authorization. I tested it with `testRegisterTableWithDeniedEmbeddedLocation`.
<img width="1082" height="143" alt="Image" src="https://github.com/user-attachments/assets/a2e2e895-10df-462a-804d-21f288f7e46f" />
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java:
##########
@@ -320,6 +325,10 @@ private LoadTableResponse loadTable(Map<String, String> vars) {
private LoadTableResponse registerTable(Map<String, String> vars, Object body) {
Namespace namespace = namespaceFromPathVars(vars);
RegisterTableRequest request = castRequest(RegisterTableRequest.class, body);
+ request.validate();
+ Map<String, String> namespaceMetadata = asNamespaceCatalog.loadNamespaceMetadata(namespace);
+ FileIO io = ((HiveCatalog) catalog).io();
Review Comment:
We don't need this FileIO if we delegate the permission check to Hive Metastore
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges drop-table-purge", e);
+ }
+ }
+
+ private void checkLocationAuthorized(String catalogName, String databaseName,
+ Map<String, String> namespaceMetadata, String location, String commandString) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, falling back to namespace containment for {}",
+ catalogName, location);
+ checkContainedInNamespace(databaseName, namespaceMetadata, location);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString(commandString);
+ try {
+ authorizer.checkPrivileges(HiveOperationType.CREATETABLE, inputs, Collections.emptyList(), builder.build());
+ } catch (HiveAccessControlException e) {
+ throw new ForbiddenException(e, e.getMessage());
+ } catch (HiveAuthzPluginException e) {
+ throw new IllegalStateException("Failed to check privileges for " + commandString, e);
+ }
+ }
+
+ private void checkContainedInNamespace(String databaseName, Map<String, String> namespaceMetadata,
+ String location) {
+ var externalRoot = namespaceMetadata.get("location");
+ if (externalRoot != null && isContained(externalRoot, location)) {
+ return;
+ }
+ if (isContained(managedNamespaceLocation(databaseName), location)) {
+ return;
+ }
+ throw new ForbiddenException(
+ "Location %s is not authorized and is not contained in namespace %s", location, databaseName);
+ }
+
+ private String managedNamespaceLocation(String databaseName) {
+ var warehouseLocation = conf.get(HiveConf.ConfVars.METASTORE_WAREHOUSE.varname);
+ Preconditions.checkNotNull(warehouseLocation, "Warehouse location is not set: hive.metastore.warehouse.dir=null");
+ if (warehouseLocation.endsWith("/")) {
+ warehouseLocation = warehouseLocation.substring(0, warehouseLocation.length() - 1);
+ }
+ return String.format("%s/%s.db", warehouseLocation, databaseName);
+ }
+
+ /**
+ * Checks whether {@code candidate} resolves under {@code root}. Both are resolved via {@link Path#toUri()} and
+ * {@link java.net.URI#normalize()}, which -- unlike {@link Path}'s own normalization -- actually collapses
+ * {@code .}/{@code ..} segments; a plain string-prefix comparison on unnormalized paths would let a location
+ * like {@code root/../../elsewhere} pass a naive check while actually resolving outside {@code root}.
+ */
+ private static boolean isContained(String root, String candidate) {
Review Comment:
Let's use `FileUtils.isPathWithinSubtree`, which is also used in [the storage handler](https://github.com/apache/hive/blob/9d9a51e103fddfeb855e46e1606b36cab770a56e/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java#L223-L225). I know there are some traps for this type of comparison (e.g., some characters are not allowed in a path in URI, but object storage allows).
https://github.com/apache/hive/blob/9d9a51e103fddfeb855e46e1606b36cab770a56e/common/src/java/org/apache/hadoop/hive/common/FileUtils.java#L1254-L1256
You may have to copy `org.apache.hadoop.hive.common.FileUtils.isPathWithinSubtree` to `org.apache.hadoop.hive.metastore.utils.FileUtils`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-6051374509
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-6077461447
Merged. @henrib Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30004: Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4075075128
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ @Override
+ public Map<String, String> properties() {
+ return delegate.properties();
+ }
+
+ @Override
+ public void initialize(Map<String, String> properties) {
+ delegate.initialize(properties);
+ }
+
+ @Override
+ public void close() {
+ delegate.close();
+ }
+
+ private static boolean isContained(String root, String candidate) {
+ return candidate.equals(root) || candidate.startsWith(root.endsWith("/") ? root : root + "/");
+ }
+
+ private static String normalize(String location) {
+ return new Path(location).toUri().normalize().toString();
+ }
Review Comment:
There are now multiple, slightly different containment/normalization implementations in the codebase (this class and `IcebergAuthorizer`). To reduce drift and security regressions over time, consider extracting a shared utility (in an appropriate common module/package) and use it in both places so containment semantics remain consistent.
##########
standalone-metastore/metastore-rest-catalog/src/test/java/org/apache/iceberg/rest/BaseRESTCatalogTests.java:
##########
@@ -277,4 +284,56 @@ void testStageCreateTableWithDeniedLocation() {
Assertions.assertThrows(ForbiddenException.class, builder::createTransaction);
Assertions.assertThrows(NoSuchTableException.class, () -> catalog.loadTable(tableIdentifier));
}
+
+ private static String writeMetadataFile(String directory, String tableLocation) throws IOException {
+ var metadataLocation = directory + "/v1.metadata.json";
+ Files.deleteIfExists(java.nio.file.Path.of(metadataLocation));
+ var io = new HadoopFileIO(new Configuration(false));
+ var metadata = TableMetadata.newTableMetadata(new Schema(), PartitionSpec.unpartitioned(), tableLocation,
+ Collections.emptyMap());
+ TableMetadataParser.write(metadata, io.newOutputFile(metadataLocation));
+ return metadataLocation;
+ }
Review Comment:
`Files.deleteIfExists(Path.of(metadataLocation))` assumes `metadataLocation` is a local filesystem path. If `directory` is ever a URI-like location (e.g., `file:/...`), `Path.of(...)` will not reference the intended file and the cleanup will be wrong. Use a URI-safe conversion (e.g., `Paths.get(URI.create(metadataLocation))` for `file:` URIs) or keep the helper consistently in local-path form and add the scheme only when passing to `FileIO`.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -241,6 +241,10 @@ public String name() {
return name;
}
+ public FileIO io() {
Review Comment:
Adding a new public accessor for the internal `FileIO` increases the public API surface of `HiveCatalog` and exposes service-level I/O capabilities to any in-process caller. Consider narrowing the exposure via an internal/spi-style interface (e.g., `SupportsFileIO`) implemented by `HiveCatalog`, or otherwise clearly documenting this method as internal-only to avoid unintended usage.
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
Review Comment:
Logging a WARN for every skipped deletion can flood logs during purge if a malicious/corrupt table references many out-of-scope files. Consider reducing verbosity (e.g., DEBUG) or rate-limiting/aggregating skipped-delete logs while still keeping enough signal for operators.
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java:
##########
@@ -320,6 +325,10 @@ private LoadTableResponse loadTable(Map<String, String> vars) {
private LoadTableResponse registerTable(Map<String, String> vars, Object body) {
Namespace namespace = namespaceFromPathVars(vars);
RegisterTableRequest request = castRequest(RegisterTableRequest.class, body);
+ request.validate();
+ Map<String, String> namespaceMetadata = asNamespaceCatalog.loadNamespaceMetadata(namespace);
+ FileIO io = ((HiveCatalog) catalog).io();
+ icebergAuthorizer.validateRegisterTable(catalogName, namespace, namespaceMetadata, request, io);
return castResponse(LoadTableResponse.class, CatalogHandlers.registerTable(catalog, namespace, request));
Review Comment:
The hard cast `((HiveCatalog) catalog)` can throw `ClassCastException` at runtime if this adapter is ever constructed with a non-`HiveCatalog` implementation. Prefer making the adapter depend on a `HiveCatalog`-typed field, or check `instanceof` and fail fast with a clear `IllegalStateException` explaining the required catalog type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4207282089
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
Review Comment:
Good catch, fixed — the location is now passed as both an input and an output in validateDropTablePurge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4185817870
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
Review Comment:
I built this branch locally, and I am checking whether DFS_URI should be validated as an input, an output, or both. I hit an issue where Iceberg REST -> HDFS access doesn't work. I'm checking why. I think this isn't a problem with HIVE-30002, but an issue on the master branch. Anyway, my review will take some more time to test it in a real Ranger environment.
```
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] fs.FileSystem: Failed to initialize filesystem hdfs://zookage/apps/hive-warehouse/test/metadata/00000-b8ac910f-489a-438d-8c67-67c748a2123e.metadata.json: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] util.Tasks: Retrying task after failure: sleepTimeMs=108 java.net.UnknownHostException: zookage
hive-metastore-server-0: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:479)
hive-metastore-server-0: at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:134)
```
BTW, have you tested it with HDFS + Ranger?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "saihemanth-cloudera (via GitHub)" <gi...@apache.org>.
saihemanth-cloudera commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4107277110
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java:
##########
@@ -320,6 +325,10 @@ private LoadTableResponse loadTable(Map<String, String> vars) {
private LoadTableResponse registerTable(Map<String, String> vars, Object body) {
Namespace namespace = namespaceFromPathVars(vars);
RegisterTableRequest request = castRequest(RegisterTableRequest.class, body);
+ request.validate();
+ Map<String, String> namespaceMetadata = asNamespaceCatalog.loadNamespaceMetadata(namespace);
+ FileIO io = ((HiveCatalog) catalog).io();
Review Comment:
This can be an HMSCachingCatalog when cache expiry is enabled, so the cast can fail at runtime. Can we get the underlying HiveCatalog another way?
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -241,6 +241,10 @@ public String name() {
return name;
}
+ public FileIO io() {
Review Comment:
+1. Exposing the catalog’s service-level FileIO publicly feels risky. Could this be package-private or behind an internal interface?
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ @Override
+ public Map<String, String> properties() {
+ return delegate.properties();
+ }
+
+ @Override
+ public void initialize(Map<String, String> properties) {
+ delegate.initialize(properties);
+ }
+
+ @Override
+ public void close() {
+ delegate.close();
+ }
+
+ private static boolean isContained(String root, String candidate) {
+ return candidate.equals(root) || candidate.startsWith(root.endsWith("/") ? root : root + "/");
Review Comment:
We now have containment logic here and in `IcebergAuthorizer`. Can we share one helper so the rules don’t drift?
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final String location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = normalize(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ if (!isContained(location, normalize(path))) {
+ LOG.warn("Skipping delete outside table location {}: {}", location, path);
Review Comment:
This could spam WARN logs if a bad manifest has lots of out-of-scope files. Maybe debug or summarize once?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4185817870
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
Review Comment:
I built this branch locally, and I am checking whether DFS_URI should be validated as an input, an output, or both. I hit an issue where Iceberg REST -> HDFS access doesn't work. I'm checking why. I think this isn't a problem with HIVE-30002, but an issue on the master branch. Anyway, my review will take some more time to test it in a real Ranger environment.
```
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] fs.FileSystem: Failed to initialize filesystem hdfs://zookage/apps/hive-warehouse/test/metadata/00000-b8ac910f-489a-438d-8c67-67c748a2123e.metadata.json: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] util.Tasks: Retrying task after failure: sleepTimeMs=108 java.net.UnknownHostException: zookage
hive-metastore-server-0: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:479)
hive-metastore-server-0: at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:134)
```
BTW, have you tested it with HDFS + Ranger?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30004: Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-5783643418
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [8 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.6% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30004: Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083904705
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
Review Comment:
I am not sure if this is needed. If a user has the DROP privilege, can he or she delete the data?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4083904705
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +175,133 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} is fetched with the
+ * catalog's shared, service-level {@link FileIO}, so both that location and the {@code location()} embedded in
+ * the metadata file it points to (which becomes the table's HMS {@code StorageDescriptor.location}, and is what
+ * a later purge trusts as its deletion root, see {@link #validateDropTablePurge}) must be authorized. Otherwise
+ * REGISTER_TABLE is an arbitrary-file-read primitive that returns any metadata file's contents to the caller.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring both locations to be contained in
+ * the namespace's external or managed root, since there is no policy to otherwise decide whether the caller may
+ * read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @param io the {@link FileIO} used to read the metadata file
+ * @throws ForbiddenException if a location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request, FileIO io) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+
+ var metadata = TableMetadataParser.read(io, request.metadataLocation());
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, metadata.location(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@link FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
Review Comment:
~~I guess this is unnecessary, since a user is typically authorized per table, not per directory.
When a user has INSERT permission on table X, they can add a new data file to the location. Similarly, with DROP permission, they can delete files in the location.
This assumption is compromised when a compromised location belongs to a table. I think CREATE TABLE or ALTER TABLE properly checks DFS_URI, so DROP TABLE can assume the location is valid.~~
I changed my mind. This could be a good idea.
https://issues.apache.org/jira/browse/HIVE-28804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4185817870
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
+ var builder = new HiveAuthzContext.Builder();
+ builder.setCommandString("drop table " + identifier.name());
+ try {
+ authorizer.checkPrivileges(HiveOperationType.DROPTABLE, inputs, Collections.emptyList(), builder.build());
Review Comment:
I built this branch locally, and I am checking whether DFS_URI should be validated as an input, an output, or both. I hit an issue where Iceberg REST -> HDFS access doesn't work. I'm checking why. I think this isn't a problem with HIVE-30002, but an issue on the master branch. Anyway, my review will take some more time.
```
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] fs.FileSystem: Failed to initialize filesystem hdfs://zookage/apps/hive-warehouse/test/metadata/00000-b8ac910f-489a-438d-8c67-67c748a2123e.metadata.json: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: 2026-10-05T15:23:33,177 WARN [qtp515442419-53] util.Tasks: Retrying task after failure: sleepTimeMs=108 java.net.UnknownHostException: zookage
hive-metastore-server-0: java.lang.IllegalArgumentException: java.net.UnknownHostException: zookage
hive-metastore-server-0: at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:479)
hive-metastore-server-0: at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:134)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4164197274
##########
iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/ScopedDeleteFileIO.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.hive;
+
+import java.util.Map;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.metastore.utils.FileUtils;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A {@link FileIO} decorator used by {@link HiveCatalog#dropTable(org.apache.iceberg.catalog.TableIdentifier,
+ * boolean)} to fence purge deletions to files under a table's own location, regardless of what a table's
+ * metadata or manifests actually reference.
+ *
+ * <p>This does not implement {@link org.apache.iceberg.io.SupportsBulkOperations} or
+ * {@link org.apache.iceberg.io.SupportsPrefixOperations} even when the delegate does, so that
+ * {@code CatalogUtil.dropTableData} is forced to route every deletion through {@link #deleteFile(String)}.
+ */
+class ScopedDeleteFileIO implements FileIO {
+ private static final Logger LOG = LoggerFactory.getLogger(ScopedDeleteFileIO.class);
+
+ private final FileIO delegate;
+ private final Path location;
+
+ ScopedDeleteFileIO(FileIO delegate, String location) {
+ this.delegate = delegate;
+ this.location = new Path(location);
+ }
+
+ @Override
+ public InputFile newInputFile(String path) {
+ return delegate.newInputFile(path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ return delegate.newOutputFile(path);
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ var candidate = new Path(path);
+ if (!FileUtils.isPathWithinSubtree(qualify(candidate, location), qualify(location, candidate))) {
+ LOG.debug("Skipping delete outside table location {}: {}", location, path);
+ return;
+ }
+ delegate.deleteFile(path);
+ }
+
+ /**
+ * A path with no scheme refers to the same filesystem location as an otherwise-identical path
+ * that does carry one, whichever side that happens to be: manifests can hold data file paths
+ * written without a scheme, and a table's own location can equally be scheme-less (e.g. when a
+ * caller supplies one explicitly). {@link org.apache.hadoop.fs.Path#equals} is scheme-sensitive,
+ * so without this a schemeless path would never match its scheme-qualified counterpart in
+ * {@link FileUtils#isPathWithinSubtree} even when it is nested directly under it. When both (or
+ * neither) sides carry a scheme, {@code path} is returned unchanged, so a genuine mismatch (e.g.
+ * a different scheme or authority) is still correctly treated as outside the table location.
+ */
+ private static Path qualify(Path path, Path other) {
+ var pathUri = path.toUri();
+ var otherUri = other.toUri();
+ if (pathUri.getScheme() == null && otherUri.getScheme() != null) {
+ return new Path(otherUri.getScheme(), otherUri.getAuthority(), pathUri.getPath());
Review Comment:
I am checking which scheme and authority to use. I expect the current implementation is fine, but we might need to use the default filesystem specified in core-site.xml. I'm checking just in case.
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FileUtils.java:
##########
@@ -658,4 +658,28 @@ void findNext() throws IOException {
nextFile = null;
}
}
+
+ /**
+ * Checks whether {@code path} is {@code subtree} itself or nested under it, by walking up
+ * {@code path}'s ancestors looking for a match. Depth-based rather than a string-prefix
+ * comparison, so it isn't fooled by sibling paths that merely share a prefix (e.g. {@code /a/bc}
+ * is not within {@code /a/b}).
+ *
+ * @param path the path to check
+ * @param subtree the candidate ancestor, or exact match
+ * @return true if {@code path} equals {@code subtree} or is one of its descendants
+ */
+ public static boolean isPathWithinSubtree(final Path path, final Path subtree) {
Review Comment:
Let's add these test cases, too.
https://github.com/apache/hive/blob/744271738e2d89f94156c41bc307efaa57a06b7e/common/src/test/org/apache/hadoop/hive/common/TestFileUtils.java#L66-L93
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4169754621
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FileUtils.java:
##########
@@ -658,4 +658,28 @@ void findNext() throws IOException {
nextFile = null;
}
}
+
+ /**
+ * Checks whether {@code path} is {@code subtree} itself or nested under it, by walking up
+ * {@code path}'s ancestors looking for a match. Depth-based rather than a string-prefix
+ * comparison, so it isn't fooled by sibling paths that merely share a prefix (e.g. {@code /a/bc}
+ * is not within {@code /a/b}).
+ *
+ * @param path the path to check
+ * @param subtree the candidate ancestor, or exact match
+ * @return true if {@code path} equals {@code subtree} or is one of its descendants
+ */
+ public static boolean isPathWithinSubtree(final Path path, final Path subtree) {
Review Comment:
Done — ported the four isPathWithinSubtree cases from common's TestFileUtils into a new metastore-common/.../utils/TestFileUtils.java.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #6812:
URL: https://github.com/apache/hive/pull/6812#discussion_r4205614266
##########
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java:
##########
@@ -161,4 +165,100 @@ void validateStageCreateTable(String catalogName, Namespace namespace, Map<Strin
throw new IllegalStateException("Failed to check privileges stage-create", e);
}
}
+
+ /**
+ * Enforces authorization for REGISTER_TABLE. The request's {@code metadataLocation} must be authorized, since
+ * REGISTER_TABLE is otherwise an arbitrary-file-read primitive that returns any metadata file's contents to the
+ * caller. The {@code location()} embedded in that metadata file (which becomes the table's HMS
+ * {@code StorageDescriptor.location}) is not checked here: it is authorized transitively by HMS's own
+ * CREATE_TABLE authorization when {@code CatalogHandlers.registerTable} creates the metastore table.
+ *
+ * <p>When no {@code HiveAuthorizer} is configured, falls back to requiring the metadata location to be
+ * contained in the namespace's external root, since there is no policy to otherwise decide whether the caller
+ * may read an arbitrary location with service credentials.
+ *
+ * @param catalogName the Hive catalog name
+ * @param namespace the Iceberg namespace
+ * @param namespaceMetadata the Iceberg namespace metadata
+ * @param request the register table request
+ * @throws ForbiddenException if the location is not authorized, or not contained in the namespace
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateRegisterTable(String catalogName, Namespace namespace, Map<String, String> namespaceMetadata,
+ RegisterTableRequest request) {
+ Preconditions.checkArgument(namespace.levels().length == 1, "Hive does not support multi-level namespaces");
+ var databaseName = namespace.level(0);
+ var commandString = "register table " + request.name();
+ checkLocationAuthorized(catalogName, databaseName, namespaceMetadata, request.metadataLocation(), commandString);
+ }
+
+ /**
+ * Enforces authorization for DROP_TABLE with {@code purge=true}. Purge deletes every file referenced by the
+ * table's current metadata using the catalog's shared, service-level {@code FileIO}, so the location must be
+ * authorized like any other DFS_URI access.
+ *
+ * <p>Unlike {@link #validateRegisterTable}, there is no namespace-containment fallback here: the structural
+ * fence in {@code HiveCatalog.dropTable} already restricts purge deletions to files under the table's own
+ * location regardless of whether a {@code HiveAuthorizer} is configured, so a deployment without one relies on
+ * that fence rather than this check.
+ *
+ * @param catalogName the Hive catalog name
+ * @param identifier the table identifier being dropped
+ * @param location the table's current location
+ * @throws ForbiddenException if the location is not authorized
+ * @throws IllegalStateException if the authorization plugin fails
+ */
+ void validateDropTablePurge(String catalogName, TableIdentifier identifier, String location) {
+ var authorizer = authorizerSupplier.get();
+ if (authorizer == null) {
+ LOG.info("No pre-event listener is configured for catalog {}, skipping drop-table-purge authorization for {}",
+ catalogName, identifier);
+ return;
+ }
+
+ var inputs = Collections.singletonList(
+ new HivePrivilegeObject(HivePrivilegeObject.HivePrivilegeObjectType.DFS_URI, location));
Review Comment:
I guess we should put this privilege object in both inputs and outputs.
As far as I've tested, DROP w/ PURGE succeeds as long as the user has READ privilege on Ranger. Given the semantics, we may also require WRITE privilege on the location.
<img width="1411" height="416" alt="Image" src="https://github.com/user-attachments/assets/ef40a897-d049-4a81-a7b2-c5cb9c733b55" />
This part is the related implementation.
https://github.com/apache/ranger/blob/edfedfeca8efe1cf7bd4372771c521731ceaca6c/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1819
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30002 : Restrict Iceberg REST catalog table registration and drop-purge to authorized/contained locations [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6812:
URL: https://github.com/apache/hive/pull/6812#issuecomment-6035763822
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6812&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6812&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6812&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6812)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org