You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "adnanhemani (via GitHub)" <gi...@apache.org> on 2026/09/22 03:03:36 UTC
[PR] Add lineage privileges and operations [polaris]
adnanhemani opened a new pull request, #5574:
URL: https://github.com/apache/polaris/pull/5574
Adds the RBAC permission model for lineage: two privileges and the three operations that consume them.
| Privilege | Code | Grantable at |
|---|---|---|
| `LINEAGE_READ` | 110 | catalog, namespace, table |
| `LINEAGE_INGEST` | 111 | catalog, namespace, table |
| Operation | Privilege | Ranger action |
|---|---|---|
| `QUERY_LINEAGE` | `LINEAGE_READ` | `lineage-query` (new) |
| `INGEST_LINEAGE` | `LINEAGE_INGEST` | `lineage-ingest` (new) |
| `REFERENCE_LINEAGE_INPUT_TABLE` | `TABLE_READ_PROPERTIES` | `table-properties-read` (existing) |
Nothing here has a call site yet โ `QUERY_LINEAGE` awaits a lineage query endpoint, and the ingest operations are consumed by ingest authorization in the OpenLineage extension, which is a separate change. The privileges have to live in `polaris-core` regardless: `PolarisPrivilege` and `PolarisAuthorizableOperation` are enums, `AuthorizationIntent` is sealed, and `RbacOperationSemantics` is package-private with a private `register()`, so an extension cannot add any of them.
### Design notes
- **Subsumption follows the described entity.** Lineage is a property *of* a table, so `LINEAGE_READ` rides its read uber-grants and `LINEAGE_INGEST` its write ones. Catalog and namespace scope inheritance comes from `hasTransitivePrivilege`'s path walk, so there are no per-scope constants and no `NAMESPACE_*` subsumption entries.
- **`LINEAGE_INGEST` does not confer `LINEAGE_READ`.** Ingest is append-only event submission; an engine principal has no need to query the graph. This departs from the `SEMANTIC_MODEL_WRITE` โ `READ` precedent, where editing presupposes reading.
- **Input datasets require `TABLE_READ_PROPERTIES`, not a lineage privilege.** Citing a table as a lineage source is not reading its lineage, and requiring `LINEAGE_INGEST` would break ordinary ETL, since ingest rides write-only uber-grants โ a job reading a source and writing a target would record no edge at all. Read access still prevents citing a table the caller cannot read. `TABLE_LIST` was rejected because `TABLE_CREATE` satisfies it.
- **All three operations use `ResolvedPathRooting.ROOT`** so root-scope grants satisfy checks.
- **Ranger mappings are included** because these operations become reachable, and `RangerPolarisAuthorizer` fail-closed denies an unmapped operation with a message indistinguishable from a real denial. Both ingest cases share one action, since an action tracks the permission concept; citing an input reuses `table-properties-read`, as `LOAD_TABLE` does, because a dedicated access type would make Ranger stricter than the default authorizer for the same operation. These names must also exist as access types in the deployment's Ranger service definition, which is not shipped here. No OPA change โ it forwards the operation name as a string.
### Testing
Two failure modes here are silent, so both have dedicated tests:
- A **missing spec literal** does not break the build; it makes `PolarisPrivilege.valueOf` throw during grant translation, returning 500 on `GET .../grants` for the whole role. `SemanticModelGrantAuthzTest` filters on `SEMANTIC_MODEL_`, so `LineageGrantAuthzTest` is the lineage analogue, round-tripping add/list/revoke at all three scopes.
- An **absent `SUPER_PRIVILEGES` key** is undetectable by the existing `subsumingPrivilegesOf` test, which derives its expectation from the same map under test. The new assertions spell the expected sets out literally; I verified by mutation that they fail when the map is widened.
`:polaris-core:test` (1027 tests), `:polaris-extensions-auth-ranger:test` (14), and `:polaris-runtime-service:test --tests '*Authz*'` (22049) all pass, along with repo-wide `compileJava`/`compileTestJava` and `spotlessCheck`. That last figure matters because `PolarisAuthzTestsFactory` builds negatives from `EnumSet.complementOf`, so the new privileges auto-generated cases across all seven authz test classes.
## Checklist
- [x] ๐ก๏ธ Don't disclose security issues! (contact security@apache.org)
- [x] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes #
- [x] ๐งช Added/updated tests with good coverage, or manually tested (and explained how)
- [x] ๐ก Added comments for complex logic
- [x] ๐งพ Updated `CHANGELOG.md` (if needed)
- [x] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add lineage privileges and operations [polaris]
Posted by "flyingImer (via GitHub)" <gi...@apache.org>.
flyingImer commented on code in PR #5574:
URL: https://github.com/apache/polaris/pull/5574#discussion_r4085618246
##########
site/content/in-dev/unreleased/managing-security/access-control.md:
##########
@@ -243,6 +243,39 @@ privileges must be granted at namespace or catalog scope.
Grant revocation uses the same request body with `POST` to the role's grants endpoint.
The existing role-grants listing returns grants with their model, namespace, or catalog scope.
+### Lineage privileges
+
+| Privilege | Target | Description |
+| --------- | ------ | ----------- |
+| LINEAGE_READ | Catalog, namespace, table | Enables querying the lineage graph for the entity. |
+| LINEAGE_INGEST | Catalog, namespace, table | Enables submitting OpenLineage events that record lineage for the entity. |
+
+Lineage is a property of the entity it describes rather than a separate child entity, so these
+privileges ride the entity's existing grants. `LINEAGE_READ` is also conferred by
+`TABLE_READ_PROPERTIES`, `TABLE_READ_DATA`, `TABLE_WRITE_PROPERTIES`, `TABLE_WRITE_DATA`,
+`TABLE_FULL_METADATA`, `CATALOG_MANAGE_METADATA`, and `CATALOG_MANAGE_CONTENT`. `LINEAGE_INGEST`
+is also conferred by the write ones: `TABLE_WRITE_PROPERTIES`, `TABLE_WRITE_DATA`,
+`TABLE_FULL_METADATA`, `CATALOG_MANAGE_METADATA`, and `CATALOG_MANAGE_CONTENT`. A principal that
+can already read a table can therefore read its lineage, and a principal that can write one can
+record lineage for it, without an additional grant.
+
+`LINEAGE_INGEST` does not confer `LINEAGE_READ`. Ingest is append-only event submission, so a
+principal granted only `LINEAGE_INGEST` cannot query the lineage graph.
+
+Granting either privilege on a catalog or namespace covers the tables beneath it, so a
+namespace-scoped grant satisfies a table-scoped check without a per-table grant.
+
+A single OpenLineage event names both the datasets a job read and the datasets it wrote, and the two
+are authorized differently. Each **output** dataset requires `LINEAGE_INGEST` on the table; when the
+output table does not exist yet, as with a CREATE TABLE AS SELECT, the check falls back to the
+parent namespace, so a principal that may create the table may also record the lineage that creates
+it. Each **input** dataset requires only read access to that table, expressed as
+`TABLE_READ_PROPERTIES`. Citing a table as a source is not reading its lineage, so `LINEAGE_READ` is
Review Comment:
nit: could we describe these checks as the intended authorization model until the ingest authorization change lands, here and in the changelog? OpenLineageAdapter still calls the provider without these checks, so the current wording makes the grants sound enforced already. The CTAS claim also needs narrowing: TABLE_CREATE does not confer LINEAGE_INGEST, and checking the parent namespace does not change the required privilege. Please make explicit that the fallback requires LINEAGE_INGEST on the namespace, directly or through a grant that subsumes it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add lineage privileges and operations [polaris]
Posted by "github-actions[bot] (via GitHub) via issues" <is...@polaris.apache.org>.
github-actions[bot] commented on PR #5574:
URL: https://github.com/apache/polaris/pull/5574#issuecomment-6647131727
This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add lineage privileges and operations [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #5574:
URL: https://github.com/apache/polaris/pull/5574#discussion_r4189580771
##########
polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizerImpl.java:
##########
@@ -730,6 +732,32 @@ public class PolarisAuthorizerImpl implements PolarisAuthorizer {
SEMANTIC_MODEL_MANAGE_GRANTS_ON_SECURABLE,
List.of(SEMANTIC_MODEL_MANAGE_GRANTS_ON_SECURABLE, CATALOG_MANAGE_ACCESS));
+ // Lineage privileges. Lineage is a property of the entity it describes, so LINEAGE_READ follows
+ // the entity's read uber-grants and LINEAGE_INGEST follows its write uber-grants. Grantability
+ // at root, catalog and namespace scope comes from hasTransitivePrivilege's path walk, not from
+ // this map. LINEAGE_INGEST deliberately does not confer LINEAGE_READ: ingest is append-only
+ // event submission, and an engine principal has no need to query the graph.
+ SUPER_PRIVILEGES.putAll(
+ LINEAGE_READ,
Review Comment:
Question: Anyone with TABLE_READ_PROPERTIES gets LINEAGE_READ for free. That's fine for the table itself. But a graph query starting at table A could return upstream or downstream tables the caller has no access to. Will the query endpoint filter neighbor nodes by the caller's own grants, or rely only on the LINEAGE_READ check on A?
##########
polaris-core/src/test/java/org/apache/polaris/core/auth/PolarisAuthorizerImplTest.java:
##########
@@ -72,6 +74,134 @@ void subsumingPrivilegesOf(PolarisPrivilege privilege) {
assertThat(actual).isEqualTo(expected);
}
+ // Expected sets are spelled out literally: subsumingPrivilegesOf falls back to
+ // EnumSet.of(privilege) for an absent key, so deriving them from SUPER_PRIVILEGES would pass
+ // whether or not the lineage keys were registered.
+
+ @Test
+ void lineageReadIsSubsumedByEntityReadAndWriteUberGrants() {
+ assertThat(PolarisAuthorizerImpl.subsumingPrivilegesOf(PolarisPrivilege.LINEAGE_READ))
+ .containsExactlyInAnyOrder(
+ PolarisPrivilege.LINEAGE_READ,
+ PolarisPrivilege.TABLE_READ_PROPERTIES,
+ PolarisPrivilege.TABLE_READ_DATA,
+ PolarisPrivilege.TABLE_WRITE_PROPERTIES,
+ PolarisPrivilege.TABLE_WRITE_DATA,
+ PolarisPrivilege.TABLE_FULL_METADATA,
+ PolarisPrivilege.CATALOG_MANAGE_METADATA,
+ PolarisPrivilege.CATALOG_MANAGE_CONTENT);
+ }
+
+ @Test
+ void lineageIngestIsSubsumedByEntityWriteUberGrants() {
+ assertThat(PolarisAuthorizerImpl.subsumingPrivilegesOf(PolarisPrivilege.LINEAGE_INGEST))
+ .containsExactlyInAnyOrder(
+ PolarisPrivilege.LINEAGE_INGEST,
+ PolarisPrivilege.TABLE_WRITE_PROPERTIES,
+ PolarisPrivilege.TABLE_WRITE_DATA,
+ PolarisPrivilege.TABLE_FULL_METADATA,
+ PolarisPrivilege.CATALOG_MANAGE_METADATA,
+ PolarisPrivilege.CATALOG_MANAGE_CONTENT);
+ }
+
+ @Test
+ void lineagePrivilegesDoNotConferEachOther() {
+ PolarisAuthorizerImpl authorizer = new PolarisAuthorizerImpl(realmConfig());
+
+ // The asymmetry is deliberate: ingest is append-only submission, so an engine principal has no
+ // need to read the graph back out. An edit that "fixes" it must justify itself here.
+ assertThat(
+ authorizer.matchesOrIsSubsumedBy(
+ PolarisPrivilege.LINEAGE_READ, PolarisPrivilege.LINEAGE_INGEST))
+ .isFalse();
+
+ // And the converse: reading the graph confers no ability to write to it.
+ assertThat(
+ authorizer.matchesOrIsSubsumedBy(
+ PolarisPrivilege.LINEAGE_INGEST, PolarisPrivilege.LINEAGE_READ))
+ .isFalse();
+ }
+
+ @Test
+ void serviceManageAccessDoesNotConferLineagePrivileges() {
+ // SERVICE_MANAGE_ACCESS is self-only as a SUPER_PRIVILEGES *key*, but it does appear as a
Review Comment:
The '26 other privileges' count goes stale the next time a privilege is added. Can we drop the number? Also, the wrap at line 131 leaves `CATALOG_MANAGE_CONTENT,` alone on a line, so can we reflow this comment?
##########
CHANGELOG.md:
##########
@@ -91,6 +91,17 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
- Semantic models now support dedicated privileges for listing, creating, reading, updating,
and dropping. Privileges can be granted to catalog roles on individual models or at namespace
or catalog scope, with separate controls for managing model grants.
+- New `LINEAGE_READ` and `LINEAGE_INGEST` privileges authorize querying the lineage graph and
+ submitting OpenLineage events, respectively. Both are grantable to catalog roles at catalog,
+ namespace, and table scope. They are also conferred by the described entity's existing grants:
+ `LINEAGE_READ` by the table read and write grants, `LINEAGE_INGEST` by the write ones.
+ `LINEAGE_INGEST` does not confer `LINEAGE_READ`. Within an OpenLineage event, output datasets
Review Comment:
This entry describes the input/output dataset checks and the CTAS namespace fallback, but none of that enforcement is in this PR. If the extension misses this release, the note describes behavior that doesn't exist. Can we keep just the privileges and their grant scopes here, and move the enforcement rules to the extension PR?
##########
polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizableOperation.java:
##########
@@ -122,6 +122,9 @@ public enum PolarisAuthorizableOperation {
LIST_SEMANTIC_MODEL,
ADD_SEMANTIC_MODEL_GRANT_TO_CATALOG_ROLE,
REVOKE_SEMANTIC_MODEL_GRANT_FROM_CATALOG_ROLE,
+ QUERY_LINEAGE,
+ INGEST_LINEAGE,
Review Comment:
Would it make sense to rename them to something like `QUERY_TABLE_LINEAGE` or `INGEST_TABLE_LINEAGE`?
##########
polaris-core/src/main/java/org/apache/polaris/core/entity/PolarisPrivilege.java:
##########
@@ -263,6 +263,8 @@ public enum PolarisPrivilege {
SEMANTIC_MODEL_DROP(107, PolarisEntityType.SEMANTIC_MODEL),
SEMANTIC_MODEL_FULL_METADATA(108, PolarisEntityType.SEMANTIC_MODEL),
SEMANTIC_MODEL_MANAGE_GRANTS_ON_SECURABLE(109, PolarisEntityType.SEMANTIC_MODEL),
+ LINEAGE_READ(110, PolarisEntityType.TABLE_LIKE),
Review Comment:
Can we rename these to `TABLE_READ_LINEAGE` / `TABLE_INGEST_LINEAGE`? Every other privilege on the `TABLE_LIKE` securable is named after its entity type (`TABLE_READ_PROPERTIES`, `TABLE_WRITE_DATA`, `VIEW_READ_PROPERTIES`, ...). The prefix names the entity, not the grant scope, so `TABLE_READ_DATA` granted on a catalog is still `TABLE_*`.
It also settles table vs. view. Tables and views share `TABLE_LIKE`, and right now `LINEAGE_*` with the `NULL_SUBTYPE` default reads as if it covers views too. With the rename plus `List.of(ICEBERG_TABLE, GENERIC_TABLE)` like `TABLE_READ_PROPERTIES`, the scope is explicit, and there's a clean `VIEW_LINEAGE_*` slot if view lineage comes later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org