You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "adam-christian-software (via GitHub)" <gi...@apache.org> on 2025/11/06 19:31:53 UTC

[PR] test: Add some Spark client tests [polaris]

adam-christian-software opened a new pull request, #3000:
URL: https://github.com/apache/polaris/pull/3000

   # Context
   As I was investigating how to support Parquet files using Generic Tables, I performed a few miscellaneous clean-ups:
   1. I did some minor README cleanup on the Spark Client. This is to streamline the reading and to incorporate the credential vending item as a known limitation.
   2. I added a JavaDoc to DeltaHelper. This was to add clarity on what the class was doing.
   3. I added some more test coverage of the Spark Client through PolarisRESTCatalogTest, DeltaHelperTest, and PolarisCatalogUtilsTest. This helped increase the code coverage in the plugin.
   4. I updated the user-facing docs for generic-table.md and polaris-spark-client.md. This documents the credential vending limitation and streamlines the content.
   
   ## Checklist
   - [X ] ๐Ÿ›ก๏ธ Don't disclose security issues! (contact security@apache.org)
   - [X ] ๐Ÿ”— Clearly explained why the changes are needed, or linked related issues: Fixes #
   - [X] ๐Ÿงช Added/updated tests with good coverage, or manually tested (and explained how)
   - [X] ๐Ÿ’ก Added comments for complex logic
   - [Not Needed] ๐Ÿงพ Updated `CHANGELOG.md` (if needed)
   - [X] ๐Ÿ“š Updated documentation in `site/content/in-dev/unreleased` (if needed)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "gh-yzou (via GitHub)" <gi...@apache.org>.
gh-yzou commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2512218855


##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -22,17 +22,17 @@ type: docs
 weight: 650
 ---
 
-Apache Polaris now provides Catalog support for Generic Tables (non-Iceberg tables), please check out
-the [Polaris Catalog OpenAPI Spec]({{% ref "polaris-api-specs/polaris-catalog-api.md" %}}) for Generic Table API specs.
+Polaris provides a Spark client to manage non-Iceberg tables through [Generic Tables]({{% ref "generic-table.md" %}}).
 
-Along with the Generic Table Catalog support, Polaris is also releasing a Spark client, which helps to
-provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
+{{< alert note >}}
+The Spark client can manage Iceberg tables and non-Iceberg tables.
 
-Note the Polaris Spark client is able to handle both Iceberg and Delta tables, not just Delta.
+Users who only use Iceberg tables can use Spark without this client.

Review Comment:
   Oh, sorry for the confusion, i was just suggesting to rephrase it like above to make it more accurate. Not really mean we should remove this description.



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Known Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data

Review Comment:
   looks good to me!



##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -119,11 +119,16 @@ If you would like to use a version of the Spark client that is currently not yet
 build a Spark client jar locally from source. Please check out the Polaris repo and refer to the Spark plugin
 [README](https://github.com/apache/polaris/blob/main/plugins/spark/README.md) for detailed instructions.
 
-## Limitations
-The Polaris Spark client has the following functionality limitations:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
+## Known Limitations

Review Comment:
   may be let's stay consistent on the titles, "Known Limitations" -> "Current Limitations"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2504607814


##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -22,17 +22,17 @@ type: docs
 weight: 650
 ---
 
-Apache Polaris now provides Catalog support for Generic Tables (non-Iceberg tables), please check out
-the [Polaris Catalog OpenAPI Spec]({{% ref "polaris-api-specs/polaris-catalog-api.md" %}}) for Generic Table API specs.
+Polaris provides a Spark client to manage non-Iceberg tables through [Generic Tables]({{% ref "generic-table.md" %}}).
 
-Along with the Generic Table Catalog support, Polaris is also releasing a Spark client, which helps to
-provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
+{{< alert note >}}
+The Spark client can manage Iceberg tables and non-Iceberg tables.
 
-Note the Polaris Spark client is able to handle both Iceberg and Delta tables, not just Delta.
+Users who only use Iceberg tables can use Spark without this client.

Review Comment:
   I 100% agree. This is to clarify to our users that they do not need this plugin to interact with Iceberg tables. I know that a user felt necessary to make this known here in Slack: https://apache-polaris.slack.com/archives/C084QSKD6S2/p1762432300234359
   
   So, this is just for clarity.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "gh-yzou (via GitHub)" <gi...@apache.org>.
gh-yzou commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2501281826


##########
plugins/spark/README.md:
##########
@@ -113,14 +117,15 @@ bin/spark-shell \
 ```
 
 # Limitations
-The Polaris Spark client supports catalog management for both Iceberg and Delta tables, it routes all Iceberg table
-requests to the Iceberg REST endpoints, and routes all Delta table requests to the Generic Table REST endpoints.
-
-The Spark Client requires at least delta 3.2.1 to work with Delta tables, which requires at least Apache Spark 3.5.3.
-Following describes the current functionality limitations of the Polaris Spark client:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
-   is also not supported, since it relies on the CTAS support.
-2) Create a Delta table without explicit location is not supported.
-3) Rename a Delta table is not supported.
-4) ALTER TABLE ... SET LOCATION is not supported for DELTA table.
-5) For other non-Iceberg tables like csv, it is not supported today.
+The following describes the current limitations of the Polaris Spark client:
+
+## General Limitations
+1. The Polaris Spark client only supports Iceberg and Delta Lake tables. It does not support other table formats like CSV, JSON, etc.
+2. Generic tables (non-Iceberg tables) do not currently support credential vending.
+
+## Delta Lake Limitations
+1. Create table as select (CTAS) is not supported for Delta Lake tables. As a result, the `saveAsTable` method of `Dataframe`

Review Comment:
   I see we updated Delta tables to Delta Lake Tables. I think Delta is the actual table metadata format, Delta lake seems more indicating the storage layer or system, and within delta lake, the table is stored in delta format. So from table format's point of view, I think Delta table is a more accurate term.



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -22,17 +22,19 @@ type: docs
 weight: 435
 ---
 
-The Generic Table in Apache Polaris is designed to provide support for non-Iceberg tables across different table formats includes delta, csv etc. It currently provides the following capabilities:
+The generic tables framework provides support for non-Iceberg table formats including Delta Lake, CSV, etc. With this framework, you can:

Review Comment:
   I don't think generic table is a framework, it is a catalog concept, can we keep what we originally have?



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Known Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data

Review Comment:
   This part explains the current contract between the server and client, can we add this part back?



##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -22,17 +22,17 @@ type: docs
 weight: 650
 ---
 
-Apache Polaris now provides Catalog support for Generic Tables (non-Iceberg tables), please check out
-the [Polaris Catalog OpenAPI Spec]({{% ref "polaris-api-specs/polaris-catalog-api.md" %}}) for Generic Table API specs.
+Polaris provides a Spark client to manage non-Iceberg tables through [Generic Tables]({{% ref "generic-table.md" %}}).
 
-Along with the Generic Table Catalog support, Polaris is also releasing a Spark client, which helps to
-provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
+{{< alert note >}}
+The Spark client can manage Iceberg tables and non-Iceberg tables.
 
-Note the Polaris Spark client is able to handle both Iceberg and Delta tables, not just Delta.
+Users who only use Iceberg tables can use Spark without this client.

Review Comment:
   -> For users who only need to interact with iceberg tables is not strictly required to use Polaris Spark Client. Regular Iceberg provided Spark Client should continue work.



##########
plugins/spark/README.md:
##########
@@ -113,14 +117,15 @@ bin/spark-shell \
 ```
 
 # Limitations

Review Comment:
   how about update this title to # Current Limitations, because all those limitations can be eventually removed, just require extra work



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2515253144


##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -119,11 +119,16 @@ If you would like to use a version of the Spark client that is currently not yet
 build a Spark client jar locally from source. Please check out the Polaris repo and refer to the Spark plugin
 [README](https://github.com/apache/polaris/blob/main/plugins/spark/README.md) for detailed instructions.
 
-## Limitations
-The Polaris Spark client has the following functionality limitations:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
+## Current Limitations
+The following describes the current limitations of the Polaris Spark client:
+
+### General Limitations
+1. The Polaris Spark client only supports Iceberg and Delta tables. It does not support other table formats like CSV, JSON, etc.
+2. Generic tables (non-Iceberg tables) do not currently support credential vending.

Review Comment:
   nit: `Generic Tables API ...`



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Current Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data
-should look like based on the metadata. For example, with the delta support, th delta log serialization, deserialization
-and update all happens at client side.
+There are some known limitations for the generic table support:
+1. Generic tables provide limited spec information. For example, there is no spec for Schema or Partition. 
+2. There is no commit coordination provided by Polaris. It is the responsibility of the engine to coordinate loading and committing data. The catalog is only aware of the generic table fields above.
+3. There is no update capability provided by Polaris. Any update to a generic table must be done through a drop and create.
+4. Generic tables do not support credential vending.

Review Comment:
   nit: `Generic Tables API does not support credential vending.`? I think we need to be clear on what is relevant to the API provided by Polaris and what belongs to table formats.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2515233417


##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -22,17 +22,17 @@ type: docs
 weight: 650
 ---
 
-Apache Polaris now provides Catalog support for Generic Tables (non-Iceberg tables), please check out
-the [Polaris Catalog OpenAPI Spec]({{% ref "polaris-api-specs/polaris-catalog-api.md" %}}) for Generic Table API specs.
+Polaris provides a Spark client to manage non-Iceberg tables through [Generic Tables]({{% ref "generic-table.md" %}}).
 
-Along with the Generic Table Catalog support, Polaris is also releasing a Spark client, which helps to
-provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
+{{< alert note >}}
+The Spark client can manage Iceberg tables and non-Iceberg tables.
 
-Note the Polaris Spark client is able to handle both Iceberg and Delta tables, not just Delta.
+Users who only use Iceberg tables can use Spark without this client.

Review Comment:
   Updated!



##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -119,11 +119,16 @@ If you would like to use a version of the Spark client that is currently not yet
 build a Spark client jar locally from source. Please check out the Polaris repo and refer to the Spark plugin
 [README](https://github.com/apache/polaris/blob/main/plugins/spark/README.md) for detailed instructions.
 
-## Limitations
-The Polaris Spark client has the following functionality limitations:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
+## Known Limitations

Review Comment:
   Updated!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software closed pull request #3000: test: Add Some Spark Client Tests and Update Documentation on Generic Tables
URL: https://github.com/apache/polaris/pull/3000


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2504617133


##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Known Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data

Review Comment:
   I agree that it is important for people to understand the contract between the client & the server. I believe that this might be helpful to put into the Generic Tables information rather than inside of the limitations. But, let me see if I can make it more concise.
   
   I workshopped it. Let me know what you think.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on PR #3000:
URL: https://github.com/apache/polaris/pull/3000#issuecomment-3499100426

   @gh-yzou & @flyrain - Here's the docs update for limitation around credential vending: https://apache-polaris.slack.com/archives/C084QSKD6S2/p1762453063740029?thread_ts=1762203273.837449&cid=C084QSKD6S2
   
   I know that @gh-yzou did a good job at answering Abed's question, so potentially, we can build on this with Yun's answers there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2515225607


##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -22,17 +22,19 @@ type: docs
 weight: 435
 ---
 
-The Generic Table in Apache Polaris is designed to provide support for non-Iceberg tables across different table formats includes delta, csv etc. It currently provides the following capabilities:
+The generic tables framework provides support for non-Iceberg table formats including Delta Lake, CSV, etc. With this framework, you can:

Review Comment:
   Assuming this is fine now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2515662092


##########
site/content/in-dev/unreleased/polaris-spark-client.md:
##########
@@ -119,11 +119,16 @@ If you would like to use a version of the Spark client that is currently not yet
 build a Spark client jar locally from source. Please check out the Polaris repo and refer to the Spark plugin
 [README](https://github.com/apache/polaris/blob/main/plugins/spark/README.md) for detailed instructions.
 
-## Limitations
-The Polaris Spark client has the following functionality limitations:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
+## Current Limitations
+The following describes the current limitations of the Polaris Spark client:
+
+### General Limitations
+1. The Polaris Spark client only supports Iceberg and Delta tables. It does not support other table formats like CSV, JSON, etc.
+2. Generic tables (non-Iceberg tables) do not currently support credential vending.

Review Comment:
   Fixed.



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Current Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data
-should look like based on the metadata. For example, with the delta support, th delta log serialization, deserialization
-and update all happens at client side.
+There are some known limitations for the generic table support:
+1. Generic tables provide limited spec information. For example, there is no spec for Schema or Partition. 
+2. There is no commit coordination provided by Polaris. It is the responsibility of the engine to coordinate loading and committing data. The catalog is only aware of the generic table fields above.
+3. There is no update capability provided by Polaris. Any update to a generic table must be done through a drop and create.
+4. Generic tables do not support credential vending.

Review Comment:
   Fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] test: Add Some Spark Client Tests and Update Documentation on Generic Tables [polaris]

Posted by "adam-christian-software (via GitHub)" <gi...@apache.org>.
adam-christian-software commented on code in PR #3000:
URL: https://github.com/apache/polaris/pull/3000#discussion_r2504616509


##########
plugins/spark/README.md:
##########
@@ -113,14 +117,15 @@ bin/spark-shell \
 ```
 
 # Limitations

Review Comment:
   That sounds good to me! Updated!



##########
plugins/spark/README.md:
##########
@@ -113,14 +117,15 @@ bin/spark-shell \
 ```
 
 # Limitations
-The Polaris Spark client supports catalog management for both Iceberg and Delta tables, it routes all Iceberg table
-requests to the Iceberg REST endpoints, and routes all Delta table requests to the Generic Table REST endpoints.
-
-The Spark Client requires at least delta 3.2.1 to work with Delta tables, which requires at least Apache Spark 3.5.3.
-Following describes the current functionality limitations of the Polaris Spark client:
-1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe`
-   is also not supported, since it relies on the CTAS support.
-2) Create a Delta table without explicit location is not supported.
-3) Rename a Delta table is not supported.
-4) ALTER TABLE ... SET LOCATION is not supported for DELTA table.
-5) For other non-Iceberg tables like csv, it is not supported today.
+The following describes the current limitations of the Polaris Spark client:
+
+## General Limitations
+1. The Polaris Spark client only supports Iceberg and Delta Lake tables. It does not support other table formats like CSV, JSON, etc.
+2. Generic tables (non-Iceberg tables) do not currently support credential vending.
+
+## Delta Lake Limitations
+1. Create table as select (CTAS) is not supported for Delta Lake tables. As a result, the `saveAsTable` method of `Dataframe`

Review Comment:
   Ahhh, ๐Ÿคฆ , you are right! Let me change back.



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -22,17 +22,19 @@ type: docs
 weight: 435
 ---
 
-The Generic Table in Apache Polaris is designed to provide support for non-Iceberg tables across different table formats includes delta, csv etc. It currently provides the following capabilities:
+The generic tables framework provides support for non-Iceberg table formats including Delta Lake, CSV, etc. With this framework, you can:

Review Comment:
   Good point! I completely agree with you that it is a catalog concept and not a framework.
   
   In my opinion, right now, it reads like a feature rather than a concept native to Polaris. I tried to match other aspects of our documentation like Policies. Let me update the text and, if it still doesn't work, maybe, we can iterate together.



##########
site/content/in-dev/unreleased/generic-table.md:
##########
@@ -157,13 +158,10 @@ curl -X DELETE http://localhost:8181/api/catalog/polaris/v1/delta_catalog/namesp
 
 For the complete and up-to-date API specification, see the [Catalog API Spec](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/generated/bundled-polaris-catalog-service.yaml).
 
-## Limitations
+## Known Limitations
 
-Current limitations of Generic Table support:
-1) Limited spec information. Currently, there is no spec for information like Schema, Partition etc.
-2) No commit coordination or update capability provided at the catalog service level.
-
-Therefore, the catalog itself is unaware of anything about the underlying table except some of the loosely defined metadata.
-It is the responsibility of the engine (and plugins used by the engine) to determine exactly how loading or committing data

Review Comment:
   I workshopped it. Let me know what you think.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org