You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "flyrain (via GitHub)" <gi...@apache.org> on 2025/04/04 18:04:00 UTC
[PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
flyrain opened a new pull request, #1314:
URL: https://github.com/apache/polaris/pull/1314
- [x] Implement PolicyCatalog Stage 1: CRUD + ListPolicies #1294
- [ ] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029496124
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
+ }
+
+ private List<PolarisEntity> getFullPath(Namespace namespace, String targetName) {
+ if (namespace == null || namespace.isEmpty()) {
+ // catalog
+ return List.of(catalogEntity);
+ } else if (Strings.isNullOrEmpty(targetName)) {
+ // namespace
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
+ if (resolvedTargetEntity == null) {
+ throw new NoSuchNamespaceException("Namespace does not exist: %s", namespace);
+ }
+ return resolvedTargetEntity.getRawFullPath();
+ } else {
+ // table
+ var tableIdentifier = TableIdentifier.of(namespace, targetName);
+ // only Iceberg tables are supported
+ var resolvedTableEntity =
+ resolvedEntityView.getResolvedPath(
+ tableIdentifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE);
+ if (resolvedTableEntity == null) {
+ throw new NoSuchTableException("Iceberg Table does not exist: %s", tableIdentifier);
+ }
+ return resolvedTableEntity.getRawFullPath();
+ }
+ }
+
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(PolicyIdentifier policyIdentifier) {
+ var resolvedEntities =
resolvedEntityView.getPassthroughResolvedPath(
policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
+ if (resolvedEntities == null || resolvedEntities.getResolvedLeafEntity() == null) {
throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
}
+ return resolvedEntities;
+ }
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
- callContext.getPolarisCallContext(),
- PolarisEntity.toCoreList(catalogPath),
- leafEntity,
- Map.of(),
- false);
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(
+ @Nonnull PolicyAttachmentTarget target) {
+ return switch (target.getType()) {
+ // get the current catalog entity, since policy cannot apply across catalog at this moment
+ case CATALOG -> resolvedEntityView.getResolvedReferenceCatalogEntity();
+ case NAMESPACE -> {
+ var namespace = Namespace.of(target.getPath().toArray(new String[0]));
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
Review Comment:
These paths will be added in `CatalogHandler` level during authorization, we will need to authorize both privileges on both policy and target.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029528647
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
A side note: I also think it's OK to populate persistence layer exception here. Everything is wrapped in a result now. I feel like this isn't necessary. It's a big change though, which is out of scope of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031151468
##########
quarkus/service/src/test/java/org/apache/polaris/service/quarkus/catalog/PolicyCatalogTest.java:
##########
@@ -112,18 +122,26 @@ public Map<String, String> getConfigOverrides() {
}
}
- protected static final Namespace NS = Namespace.of("newdb");
- protected static final TableIdentifier TABLE = TableIdentifier.of(NS, "table");
- public static final String CATALOG_NAME = "polaris-catalog";
- public static final String TEST_ACCESS_KEY = "test_access_key";
- public static final String SECRET_ACCESS_KEY = "secret_access_key";
- public static final String SESSION_TOKEN = "session_token";
-
- private static final Namespace NS1 = Namespace.of("ns1");
- private static final PolicyIdentifier POLICY1 = new PolicyIdentifier(NS1, "p1");
- private static final PolicyIdentifier POLICY2 = new PolicyIdentifier(NS1, "p2");
- private static final PolicyIdentifier POLICY3 = new PolicyIdentifier(NS1, "p3");
- private static final PolicyIdentifier POLICY4 = new PolicyIdentifier(NS1, "p4");
+ private static final Namespace NS = Namespace.of("ns1");
Review Comment:
Those are all useful constants, why this change to make those `private`?
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
Review Comment:
Just yielding a `boolean` makes it impossible to determine the cause of the error.
It's inconsistent behavior that some functions throw exceptions and some just return `false` in case of an error (and hiding the cause).
##########
polaris-core/src/main/java/org/apache/polaris/core/persistence/transactional/TransactionalMetaStoreManagerImpl.java:
##########
@@ -47,6 +47,7 @@
import org.apache.polaris.core.entity.PolarisPrivilege;
import org.apache.polaris.core.entity.PolarisTaskConstants;
import org.apache.polaris.core.persistence.*;
+import org.apache.polaris.core.persistence.PolicyMappingAlreadyExistsException;
Review Comment:
Why this change?
BTW: wildcard imports shouldn't be used.
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
Review Comment:
What if this returns `false`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029560502
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
> How about letting it extends PolarisException? I'm trying to avoid adding a new exception. I'm fine with reuse this one. I was also thinking about move this exception into the package policy.exceptions, but defer to a followup for that.
Sounds good, let's first extend the PolarisException and move in a later PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029503484
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
Review Comment:
Do we want to convert `PolicyEntity` to `Policy` for every policy fetched in this step? I think we can directly return a list of `PolicyEntity` here and defer the conversion till we have a final list of effective `PolicyEntity`. WDYT?
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
I think we need to define a new exception that extends `PolarisException` instead of using the one defined in the persistence. Otherwise, it won't be mapped to 409 in http response
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031826511
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -249,27 +246,246 @@ public Policy updatePolicy(
return constructPolicy(newPolicyEntity);
}
- public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
+ public void dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
+ var result =
metaStoreManager.dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
false);
- return dropEntityResult.isSuccess();
+ if (!result.isSuccess()) {
Review Comment:
> From the user POV, the return of this method doesn't matter but rather the return code (and content) of the REST API matters.
Not sure I understand completely. The http return code is controlled by the exception mapper, like `IcebergExceptionMapper` and `PolarisExceptionMapper`.
> throwing an exception here looks incorrect to me, or at least inconsistent with IcebergCatalog and GenericTableCatalog
It is inconsistent with a method like `public boolean dropTable(TableIdentifier tableIdentifier, boolean purge)` in `IcebergCatalog`, but I believe it's the right thing to do. The `dropTable` method needs to return a boolean only because it is inherited from the method in a class from Iceberg lib. I didn't see any reason we need a boolean return. Or am I missing something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031690547
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
Review Comment:
That's a good point. Fixed in a new commit.
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
Review Comment:
Fixed in a new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2032160427
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -249,27 +246,246 @@ public Policy updatePolicy(
return constructPolicy(newPolicyEntity);
}
- public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
+ public void dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
+ var result =
metaStoreManager.dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
false);
- return dropEntityResult.isSuccess();
+ if (!result.isSuccess()) {
Review Comment:
Added back boolean return per offline discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029451818
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
+ }
+
+ private List<PolarisEntity> getFullPath(Namespace namespace, String targetName) {
+ if (namespace == null || namespace.isEmpty()) {
+ // catalog
+ return List.of(catalogEntity);
+ } else if (Strings.isNullOrEmpty(targetName)) {
+ // namespace
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
+ if (resolvedTargetEntity == null) {
+ throw new NoSuchNamespaceException("Namespace does not exist: %s", namespace);
+ }
+ return resolvedTargetEntity.getRawFullPath();
+ } else {
+ // table
+ var tableIdentifier = TableIdentifier.of(namespace, targetName);
+ // only Iceberg tables are supported
+ var resolvedTableEntity =
+ resolvedEntityView.getResolvedPath(
+ tableIdentifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE);
+ if (resolvedTableEntity == null) {
+ throw new NoSuchTableException("Iceberg Table does not exist: %s", tableIdentifier);
+ }
+ return resolvedTableEntity.getRawFullPath();
+ }
+ }
+
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(PolicyIdentifier policyIdentifier) {
+ var resolvedEntities =
resolvedEntityView.getPassthroughResolvedPath(
policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
+ if (resolvedEntities == null || resolvedEntities.getResolvedLeafEntity() == null) {
throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
}
+ return resolvedEntities;
+ }
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
- callContext.getPolarisCallContext(),
- PolarisEntity.toCoreList(catalogPath),
- leafEntity,
- Map.of(),
- false);
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(
+ @Nonnull PolicyAttachmentTarget target) {
+ return switch (target.getType()) {
+ // get the current catalog entity, since policy cannot apply across catalog at this moment
+ case CATALOG -> resolvedEntityView.getResolvedReferenceCatalogEntity();
+ case NAMESPACE -> {
+ var namespace = Namespace.of(target.getPath().toArray(new String[0]));
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
Review Comment:
IIUC, we will need to create a `PolicyCatalog` object [here](https://github.com/polaris-catalog/polaris/blob/main/service/common/src/main/java/org/apache/polaris/service/context/PolarisCallContextCatalogFactory.java#L86-L86), just like what we did for `IcebergCatalog`. cc @HonahX as he created this class in #1294.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029528647
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
A side note: I also think it's OK to populate persistence layer exception here. Everything is wrapped in a result now. I feel like this isn't necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031725377
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -249,27 +246,246 @@ public Policy updatePolicy(
return constructPolicy(newPolicyEntity);
}
- public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
+ public void dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
+ var result =
metaStoreManager.dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
false);
- return dropEntityResult.isSuccess();
+ if (!result.isSuccess()) {
Review Comment:
throwing an exception here looks incorrect to me, or at least inconsistent with IcebergCatalog and GenericTableCatalog. From the user POV, the return of this method doesn't matter but rather the return code (and content) of the REST API matters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031705879
##########
polaris-core/src/main/java/org/apache/polaris/core/persistence/transactional/TransactionalMetaStoreManagerImpl.java:
##########
@@ -47,6 +47,7 @@
import org.apache.polaris.core.entity.PolarisPrivilege;
import org.apache.polaris.core.entity.PolarisTaskConstants;
import org.apache.polaris.core.persistence.*;
+import org.apache.polaris.core.persistence.PolicyMappingAlreadyExistsException;
Review Comment:
Good catch! Revert the change.
##########
polaris-core/src/main/java/org/apache/polaris/core/persistence/transactional/TransactionalMetaStoreManagerImpl.java:
##########
@@ -47,6 +47,7 @@
import org.apache.polaris.core.entity.PolarisPrivilege;
import org.apache.polaris.core.entity.PolarisTaskConstants;
import org.apache.polaris.core.persistence.*;
+import org.apache.polaris.core.persistence.PolicyMappingAlreadyExistsException;
Review Comment:
Good catch! Reverted the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029712098
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
Review Comment:
Good idea! Done in the new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029712057
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
Done in the new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029527803
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
Review Comment:
How about letting it extends PolarisException? I'm trying to avoid adding a new exception. I'm fine with reuse this one. I was also thinking about move this exception into the package policy.exceptions, but defer to a followup for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2032103658
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -249,27 +246,246 @@ public Policy updatePolicy(
return constructPolicy(newPolicyEntity);
}
- public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
+ public void dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
+ var result =
metaStoreManager.dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
false);
- return dropEntityResult.isSuccess();
+ if (!result.isSuccess()) {
Review Comment:
> Not sure I understand completely. The http return code is controlled by the exception mapper, like IcebergExceptionMapper and PolarisExceptionMapper.
This is true, but this method is not the final touchpoint between the business logic and the REST logic. From what I understand, there will be another layer on top of this like `IcebergCatalogAdapter`, right?
> I didn't see any reason we need a boolean return. Or am I missing something?
We fundamentally have a boolean outcome here -- the drop either succeeds or it does not. We could use an exception for flow control here, but if we can achieve the same outcome with a boolean shouldn't we prefer that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain merged PR #1314:
URL: https://github.com/apache/polaris/pull/1314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029424898
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
+ }
+
+ private List<PolarisEntity> getFullPath(Namespace namespace, String targetName) {
+ if (namespace == null || namespace.isEmpty()) {
+ // catalog
+ return List.of(catalogEntity);
+ } else if (Strings.isNullOrEmpty(targetName)) {
+ // namespace
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
+ if (resolvedTargetEntity == null) {
+ throw new NoSuchNamespaceException("Namespace does not exist: %s", namespace);
+ }
+ return resolvedTargetEntity.getRawFullPath();
+ } else {
+ // table
+ var tableIdentifier = TableIdentifier.of(namespace, targetName);
+ // only Iceberg tables are supported
+ var resolvedTableEntity =
+ resolvedEntityView.getResolvedPath(
+ tableIdentifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE);
+ if (resolvedTableEntity == null) {
+ throw new NoSuchTableException("Iceberg Table does not exist: %s", tableIdentifier);
+ }
+ return resolvedTableEntity.getRawFullPath();
+ }
+ }
+
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(PolicyIdentifier policyIdentifier) {
+ var resolvedEntities =
resolvedEntityView.getPassthroughResolvedPath(
policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
+ if (resolvedEntities == null || resolvedEntities.getResolvedLeafEntity() == null) {
throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
}
+ return resolvedEntities;
+ }
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
- callContext.getPolarisCallContext(),
- PolarisEntity.toCoreList(catalogPath),
- leafEntity,
- Map.of(),
- false);
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(
+ @Nonnull PolicyAttachmentTarget target) {
+ return switch (target.getType()) {
+ // get the current catalog entity, since policy cannot apply across catalog at this moment
+ case CATALOG -> resolvedEntityView.getResolvedReferenceCatalogEntity();
+ case NAMESPACE -> {
+ var namespace = Namespace.of(target.getPath().toArray(new String[0]));
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
Review Comment:
Where do these paths get added to the resolver?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2029496124
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +248,224 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .dropEntityIfExists(
+ callContext.getPolarisCallContext(),
+ PolarisEntity.toCoreList(catalogPath),
+ policyEntity,
+ Map.of(),
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
+ }
+
+ public List<Policy> getApplicablePolicies(
+ Namespace namespace, String targetName, PolicyType policyType) {
+ var targetFullPath = getFullPath(namespace, targetName);
+ return getEffectivePolicies(targetFullPath, policyType);
+ }
+
+ /**
+ * Returns the effective policies for a given hierarchical path and policy type.
+ *
+ * <p>Potential Performance Improvements:
+ *
+ * <ul>
+ * <li>Range Query Optimization: Enhance the query mechanism to fetch policies for all entities
+ * in a single range query, reducing the number of individual queries against the mapping
+ * table.
+ * <li>Filtering on Inheritable: Improve the filtering process by applying the inheritable
+ * condition at the data retrieval level, so that only the relevant policies for non-leaf
+ * nodes are processed.
+ * <li>Caching: Implement caching for up-level policies to avoid redundant calculations and
+ * lookups, especially for frequently accessed paths.
+ * </ul>
+ *
+ * @param path the list of entities representing the hierarchical path
+ * @param policyType the type of policy to filter on
+ * @return a list of effective policies, combining inherited policies from upper levels and
+ * non-inheritable policies from the final entity
+ */
+ private List<Policy> getEffectivePolicies(List<PolarisEntity> path, PolicyType policyType) {
+ if (path == null || path.isEmpty()) {
+ return List.of();
+ }
+
+ Map<String, Policy> inheritedPolicies = new LinkedHashMap<>();
+ // Final list of effective policies (inheritable + last-entity non-inheritable)
+ List<Policy> finalPolicies = new ArrayList<>();
+
+ // Process all entities except the last one
+ for (int i = 0; i < path.size() - 1; i++) {
+ PolarisEntity entity = path.get(i);
+ List<Policy> currentPolicies = getPolicies(entity, policyType);
+
+ for (Policy policy : currentPolicies) {
+ // For non-last entities, we only carry forward inheritable policies
+ if (policy.getInheritable()) {
+ // Put in map; overwrites by policyType if encountered again
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ }
+ }
+ }
+
+ // Now handle the last entity's policies
+ List<Policy> lastPolicies = getPolicies(path.getLast(), policyType);
+
+ for (Policy policy : lastPolicies) {
+ if (policy.getInheritable()) {
+ // Overwrite anything by the same policyType in the inherited map
+ inheritedPolicies.put(policy.getPolicyType(), policy);
+ } else {
+ // Non-inheritable => goes directly to final list
+ finalPolicies.add(policy);
+ }
+ }
+
+ // Append all inherited policies at the end, preserving insertion order
+ finalPolicies.addAll(inheritedPolicies.values());
+
+ return finalPolicies;
+ }
+
+ private List<Policy> getPolicies(PolarisEntity target, PolicyType policyType) {
+ LoadPolicyMappingsResult result;
+ if (policyType == null) {
+ result = metaStoreManager.loadPoliciesOnEntity(callContext.getPolarisCallContext(), target);
+ } else {
+ result =
+ metaStoreManager.loadPoliciesOnEntityByType(
+ callContext.getPolarisCallContext(), target, policyType);
+ }
+
+ return result.getEntities().stream().map(PolicyCatalog::toPolicy).toList();
+ }
+
+ private List<PolarisEntity> getFullPath(Namespace namespace, String targetName) {
+ if (namespace == null || namespace.isEmpty()) {
+ // catalog
+ return List.of(catalogEntity);
+ } else if (Strings.isNullOrEmpty(targetName)) {
+ // namespace
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
+ if (resolvedTargetEntity == null) {
+ throw new NoSuchNamespaceException("Namespace does not exist: %s", namespace);
+ }
+ return resolvedTargetEntity.getRawFullPath();
+ } else {
+ // table
+ var tableIdentifier = TableIdentifier.of(namespace, targetName);
+ // only Iceberg tables are supported
+ var resolvedTableEntity =
+ resolvedEntityView.getResolvedPath(
+ tableIdentifier, PolarisEntityType.TABLE_LIKE, PolarisEntitySubType.ICEBERG_TABLE);
+ if (resolvedTableEntity == null) {
+ throw new NoSuchTableException("Iceberg Table does not exist: %s", tableIdentifier);
+ }
+ return resolvedTableEntity.getRawFullPath();
+ }
+ }
+
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(PolicyIdentifier policyIdentifier) {
+ var resolvedEntities =
resolvedEntityView.getPassthroughResolvedPath(
policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
+ if (resolvedEntities == null || resolvedEntities.getResolvedLeafEntity() == null) {
throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
}
+ return resolvedEntities;
+ }
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
-
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
- callContext.getPolarisCallContext(),
- PolarisEntity.toCoreList(catalogPath),
- leafEntity,
- Map.of(),
- false);
+ private PolarisResolvedPathWrapper getResolvedPathWrapper(
+ @Nonnull PolicyAttachmentTarget target) {
+ return switch (target.getType()) {
+ // get the current catalog entity, since policy cannot apply across catalog at this moment
+ case CATALOG -> resolvedEntityView.getResolvedReferenceCatalogEntity();
+ case NAMESPACE -> {
+ var namespace = Namespace.of(target.getPath().toArray(new String[0]));
+ var resolvedTargetEntity = resolvedEntityView.getResolvedPath(namespace);
Review Comment:
These paths will be added in `CatalogHandler` level during authorization, we will need to authorize privileges on both policy and target.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031612460
##########
quarkus/service/src/test/java/org/apache/polaris/service/quarkus/catalog/PolicyCatalogTest.java:
##########
@@ -112,18 +122,26 @@ public Map<String, String> getConfigOverrides() {
}
}
- protected static final Namespace NS = Namespace.of("newdb");
- protected static final TableIdentifier TABLE = TableIdentifier.of(NS, "table");
- public static final String CATALOG_NAME = "polaris-catalog";
- public static final String TEST_ACCESS_KEY = "test_access_key";
- public static final String SECRET_ACCESS_KEY = "secret_access_key";
- public static final String SESSION_TOKEN = "session_token";
-
- private static final Namespace NS1 = Namespace.of("ns1");
- private static final PolicyIdentifier POLICY1 = new PolicyIdentifier(NS1, "p1");
- private static final PolicyIdentifier POLICY2 = new PolicyIdentifier(NS1, "p2");
- private static final PolicyIdentifier POLICY3 = new PolicyIdentifier(NS1, "p3");
- private static final PolicyIdentifier POLICY4 = new PolicyIdentifier(NS1, "p4");
+ private static final Namespace NS = Namespace.of("ns1");
Review Comment:
They are useful, but only used within the current class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031690547
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
+ }
+
+ public boolean detachPolicy(PolicyIdentifier policyIdentifier, PolicyAttachmentTarget target) {
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ return metaStoreManager
+ .detachPolicyFromEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity)
+ .isSuccess();
Review Comment:
That's a good point. Let me fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement PolicyCatalog Stage 2: detach/attach/getApplicablePolicies [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #1314:
URL: https://github.com/apache/polaris/pull/1314#discussion_r2031690973
##########
service/common/src/main/java/org/apache/polaris/service/catalog/policy/PolicyCatalog.java:
##########
@@ -251,25 +247,219 @@ public Policy updatePolicy(
public boolean dropPolicy(PolicyIdentifier policyIdentifier, boolean detachAll) {
// TODO: Implement detachAll when we support attach/detach policy
- PolarisResolvedPathWrapper resolvedEntities =
- resolvedEntityView.getPassthroughResolvedPath(
- policyIdentifier, PolarisEntityType.POLICY, PolarisEntitySubType.NULL_SUBTYPE);
- if (resolvedEntities == null) {
- throw new NoSuchPolicyException(String.format("Policy does not exist: %s", policyIdentifier));
- }
-
- List<PolarisEntity> catalogPath = resolvedEntities.getRawParentPath();
- PolarisEntity leafEntity = resolvedEntities.getRawLeafEntity();
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var catalogPath = resolvedPolicyPath.getRawParentPath();
+ var policyEntity = resolvedPolicyPath.getRawLeafEntity();
- DropEntityResult dropEntityResult =
- metaStoreManager.dropEntityIfExists(
+ return metaStoreManager
+ .dropEntityIfExists(
callContext.getPolarisCallContext(),
PolarisEntity.toCoreList(catalogPath),
- leafEntity,
+ policyEntity,
Map.of(),
- false);
+ false)
+ .isSuccess();
+ }
+
+ public boolean attachPolicy(
+ PolicyIdentifier policyIdentifier,
+ PolicyAttachmentTarget target,
+ Map<String, String> parameters) {
+
+ var resolvedPolicyPath = getResolvedPathWrapper(policyIdentifier);
+ var policyCatalogPath = PolarisEntity.toCoreList(resolvedPolicyPath.getRawParentPath());
+ var policyEntity = PolicyEntity.of(resolvedPolicyPath.getRawLeafEntity());
+
+ var resolvedTargetPath = getResolvedPathWrapper(target);
+ var targetCatalogPath = PolarisEntity.toCoreList(resolvedTargetPath.getRawParentPath());
+ var targetEntity = resolvedTargetPath.getRawLeafEntity();
+
+ PolicyValidators.validateAttach(policyEntity, targetEntity);
+
+ var result =
+ metaStoreManager.attachPolicyToEntity(
+ callContext.getPolarisCallContext(),
+ targetCatalogPath,
+ targetEntity,
+ policyCatalogPath,
+ policyEntity,
+ parameters);
+
+ if (result.getReturnStatus() == POLICY_MAPPING_OF_SAME_TYPE_ALREADY_EXISTS) {
+ var targetId = catalogEntity.getName();
+ if (target.getType() != CATALOG) {
+ targetId += "." + String.join(".", target.getPath());
+ }
+ throw new PolicyMappingAlreadyExistsException(
+ "The policy mapping of same type(%s) for %s already exists",
+ policyEntity.getPolicyType().getName(), targetId);
+ }
+
+ return result.isSuccess();
Review Comment:
Will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org