You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "MonkeyCanCode (via GitHub)" <gi...@apache.org> on 2026/03/16 03:12:11 UTC
[PR] CLI: Add summary subcommand [polaris]
MonkeyCanCode opened a new pull request, #4003:
URL: https://github.com/apache/polaris/pull/4003
<!--
๐ Describe what changes you're proposing, especially breaking or user-facing changes.
๐ See https://github.com/apache/polaris/blob/main/CONTRIBUTING.md for more.
-->
Per [ML](https://lists.apache.org/thread/35zzzh2jgorhx7q2xksp7rwxnt6gl2zx), this PR adds some basic summary/statistics for Polaris entities which can make the CLI more useful for day to day operations. As agreed on the ML, we will skip anything that involves FileIO to keep this lightweight. We can come back later to add more interesting stats if desired.
For this PR, I added `summary` for `catalogs`, `catalog-roles`, `principals`, `principal-roles`, `namespaces`, and `policies` (`privileges` class doesn't have `summary` subcommand as it doesn't make much to add it there as it is covered by other entities already from this PR).
Setup:
```
./polaris --profile dev setup apply site/content/guides/assets/polaris/reference-setup-config.yaml
```
Sample output - catalogs:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev catalogs summary quickstart_catalog
Catalog: quickstart_catalog
--------------------------------------------------------------------------------
Metadata
Type: INTERNAL
Storage Type: FILE
Base Location: file:///var/tmp/quickstart_catalog/
Allowed Locations: file:///var/tmp/quickstart_catalog/
Created: 2026-03-16 02:08:03 UTC
Modified: 2026-03-16 02:08:03 UTC
Version: 1
Inventory
Namespaces: 2
Tables: 0
Views: 0
Access Control
Catalog Roles: 4
Assigned Principal Roles: 4
Applicable Policies
- orphan-file-policy
--------------------------------------------------------------------------------
```
Sample output - catalog-roles:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev catalog-roles summary --catalog quickstart_catalog dev_catalog_role
Catalog Role: dev_catalog_role [Catalog: quickstart_catalog]
--------------------------------------------------------------------------------
Metadata
Created: 2026-03-16 02:08:03 UTC
Modified: 2026-03-16 02:08:03 UTC
Version: 1
Assignments
Assigned Principal Roles: 1
Total Grants: 3
Privileges
- TABLE_FULL_METADATA on dev_namespace
- TABLE_READ_DATA on dev_namespace
- TABLE_WRITE_DATA on dev_namespace
--------------------------------------------------------------------------------
```
Sample output - principals:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev principals summary dev_user
Principal: dev_user
--------------------------------------------------------------------------------
Metadata
Client ID: 25a74cefb3da5fdb
Created: 2026-03-16 02:08:03 UTC
Modified: 2026-03-16 02:08:03 UTC
Version: 1
Assigned Roles
- dev_user_role
Accessible Catalogs
- quickstart_catalog
--------------------------------------------------------------------------------
```
Sample output - principal-roles:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev principal-roles summary dev_user_role
Principal Role: dev_user_role
--------------------------------------------------------------------------------
Metadata
Created: 2026-03-16 02:08:03 UTC
Modified: 2026-03-16 02:08:03 UTC
Version: 1
Assigned Principals
- dev_user
Assigned Catalog Roles
- dev_catalog_role (Catalog: quickstart_catalog)
--------------------------------------------------------------------------------
```
Sample output - namespaces:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev namespaces summary --catalog quickstart_catalog dev_namespace.sub_namespace
Namespace: dev_namespace.sub_namespace
--------------------------------------------------------------------------------
Metadata
Level: 2
Parent: dev_namespace
Sub-namespaces: 0
Tables: 0
Views: 0
Effective Policies
- orphan-file-policy (Inherited from dev_namespace)
- snapshot-expiry-policy (Inherited from dev_namespace)
--------------------------------------------------------------------------------
```
Sample output - policies:
```
โ polaris git:(cli_summary_subcommand) ./polaris --profile dev policies summary --catalog quickstart_catalog --namespace dev_namespace orphan-file-policy
Policy: orphan-file-policy
--------------------------------------------------------------------------------
Metadata
Type: system.orphan-file-removal
Description: Policy to remove orphan files
Inheritable: True
Content: {
"version": "2025-02-03",
"enable": true,
"max_orphan_file_age_in_days": 30,
"locations": [
"s3://my-bucket/my-table-location"
],
"config": {
"prefix_mismatch_mode": "ignore",
"key1": "value1"
}
}
Metadata Version: 0
--------------------------------------------------------------------------------
```
## Checklist
- [x] ๐ก๏ธ Don't disclose security issues! (contact security@apache.org)
- [x] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes #
- [x] ๐งช Added/updated tests with good coverage, or manually tested (and explained how)
- [x] ๐ก Added comments for complex logic
- [x] ๐งพ Updated `CHANGELOG.md` (if needed)
- [x] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summarize subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on PR #4003:
URL: https://github.com/apache/polaris/pull/4003#issuecomment-4072116699
> Some idea to throw: do we think about summarize a Polaris entity by just providing its identifier. It could be useful when we got an identifier but not knowing its type(table/view/policy). Not a blocker.
That is a great idea. Finding out what an entity is can be challenge especially when there are multiple of them using same name but different entity type. It would be nice to have an option to do "polaris summary IDENTIFIER" then we perform some sort of fuzzy search on this then display all related info for that given IDENTIFIER (e.g. an end-user may have principal name be the same as principal role name when doing one to one mapping and got a request to find out all related entities to a IDENTIFIER such as `sales`). However, the current `summarize` in this PR is a sub-command so user will have to know what kind the resource is as that will be from the main command.
To support this, we can introduce `summarize` as a main command then have option of `get|find|lookup` for a given entity based on fuzzy IDENTIFIER, once a fuzzy match is found, it should then display the result of the `summarize` sub-command for all matched. I have couple more things I would like to add into this in phase two. So I can for sure pick this up along with that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summarize subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode merged PR #4003:
URL: https://github.com/apache/polaris/pull/4003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944132530
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
+
+```
+input: polaris catalogs summary --help
+options:
+ summary
+ Positional arguments:
+ catalog
+```
+
+##### Examples
+
+```
+polaris catalogs stats some_catalog
Review Comment:
Good catch. Yes, I started with `stats` then changed to `summary`. Updated to `summarize` now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summarize subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944130734
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
Review Comment:
+1 for using `summarize` over current `summary`
IMO, the current `get` only display the fact/metadata about a given resource (taking `catalogs get xxxx` as an example, it will get us the catalog properties for the requested catalog but not provide us high level metrics/stats such as num of entities within this catalog). So the idea here is to give a high level view of a given resources and its reference via `summarize`. All of those are achieve-able via REST or using SQL via compute engines, but the idea here is why do i need to get to a diff tool or write another one-off script if the CLI can provide us those important info already.
The ones I have current is still basic as phrase one, but I think we can extend this and provide more quick insights when people are doing admin work on check the inventory and relation info among entities within Polaris.
Among those summarize sub-commands, I think the only one which doesn't do those enhancement is `policies` as those are pretty much the same as the one we have in `get`. This is because there is no api for us to collect those stats without traversal through all ns, table, and views (which can be expensive but something that we can add if desired). Maybe we should drop the `summarize` for `policies`? (already removed for now, maybe we can add more interesting stuff on phase two instead to avoid the confusion with standard `get`).
I am not sure if we should merge this with `get` as get should be getting me the info of a given entity instead of showing me all of the relations for a given entity. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2943745838
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
+
+```
+input: polaris catalogs summary --help
+options:
+ summary
+ Positional arguments:
+ catalog
+```
+
+##### Examples
+
+```
+polaris catalogs stats some_catalog
Review Comment:
stats -> summary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2943743748
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
Review Comment:
Minor: What's the different between `get` a Polaris entity and `summarize` one? Does it mean that `summarize` provides more informations? For example, when we get a catalog, we never get its inventory and applied policies, etc. Also maybe we should move this close to the `get`, so that people can understand both better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on PR #4003:
URL: https://github.com/apache/polaris/pull/4003#issuecomment-4071584247
Some idea to throw: do we think about summarize a Polaris entity by just providing its identifier. It could be useful when we got an identifier but not knowing its type(table/view/policy). Not a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944136757
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
+
+```
+input: polaris catalogs summary --help
+options:
+ summary
+ Positional arguments:
+ catalog
Review Comment:
Do you mind be more specific? Currently this is only for summarize a target catalog, which fields are you referring to here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summarize subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944130734
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
Review Comment:
+1 for using `summarize` over current `summary`
IMO, the current `get` only display the fact/metadata about a given resource (taking `catalogs get xxxx` as an example, it will get us the catalog properties for the requested catalog but not provide us high level metrics/stats such as num of entities within this catalog). So the idea here is to give a high level view of a given resources and its reference via `summarize`. All of those are achieve-able via REST or using SQL via compute engines, but the idea here is why do i need to get to a diff tool or write another one-off script if the CLI can provide us those important info already.
The ones I have current is still basic as phrase one, but I think we can extend this and provide more quick insights when people are doing admin work on check the inventory and relation info among entities within Polaris.
Among those summarize sub-commands, I think the only one which doesn't do those enhancement is `policies` as those are pretty much the same as the one we have in `get`. This is because there is no api for us to collect those stats without traversal through all ns, table, and views (which can be expensive but something that we can add if desired). Maybe we should drop the `summarize` for `policies`?
I am not sure if we should merge this with `get` as get should be getting me the info of a given entity instead of showing me all of the relations for a given entity. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2943752059
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -469,6 +488,24 @@ polaris principals reset --new-client-secret ${NEW_CLIENT_SECRET} some_user
polaris principals reset --new-client-id ${NEW_CLIENT_ID} --new-client-secret ${NEW_CLIENT_SECRET} some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a principal.
+
+```
+input: polaris principals summary --help
+options:
+ summary
+ Positional arguments:
+ principal
Review Comment:
same here for adding supported fields
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2943748793
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
+
+```
+input: polaris catalogs summary --help
+options:
+ summary
+ Positional arguments:
+ catalog
Review Comment:
I'd suggest to provide all fields it supports ATM, including inventory etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2943738666
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
Review Comment:
Should we use a verb here? like `summarize` or `sum` for short.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944130734
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -268,6 +269,24 @@ polaris catalogs list
polaris catalogs list --principal-role some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a catalog.
Review Comment:
+1 for using `summarize` over current `summary`
IMO, the current `get` only display the fact/metadata about a given resource (taking `catalogs get xxxx` as an example, it will get us the catalog properties for the requested catalog but not provide us high level metrics/stats such as num of entities within this catalog). So the idea here is to give a high level view of a given resources and its reference via `summarize`. All of those are achieve-able via REST or using SQL via compute engines, but the idea here is why do i need to get to a diff tool or write another one-off script if the CLI can provide us those important info already.
The ones I have current is still basic as phrase one, but I think we can extend this and provide more quick insights when people are doing admin work on check the inventory and relation info among entities within Polaris.
I am not sure if we should merge this with `get` as get should be getting me the info of a given entity instead of showing me all of the relations for a given entity. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLI: Add summary subcommand [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #4003:
URL: https://github.com/apache/polaris/pull/4003#discussion_r2944137440
##########
site/content/in-dev/unreleased/command-line-interface.md:
##########
@@ -469,6 +488,24 @@ polaris principals reset --new-client-secret ${NEW_CLIENT_SECRET} some_user
polaris principals reset --new-client-id ${NEW_CLIENT_ID} --new-client-secret ${NEW_CLIENT_SECRET} some_user
```
+#### summary
+
+The `summary` subcommand is used to display summary for a principal.
+
+```
+input: polaris principals summary --help
+options:
+ summary
+ Positional arguments:
+ principal
Review Comment:
Same as above. I would need a bit more clarification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org