You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "sathesuraj (via GitHub)" <gi...@apache.org> on 2025/08/28 11:19:10 UTC
[I] Support Cockroach DB as Metastore [polaris]
sathesuraj opened a new issue, #2464:
URL: https://github.com/apache/polaris/issues/2464
### Is your feature request related to a problem? Please describe.
Currently, Polaris does not support CockroachDB. It only supports PostgreSQL, even though CockroachDB is highly compatible with PostgreSQL.
### Describe the solution you'd like
CockroachDB uses INT8 (64-bit signed integer) as the default storage type for INT, INTEGER, and BIGINT data types. It does not support a 32-bit integer type (INT4) as a separate storage type. This design simplifies type handling and avoids integer overflow issues.
In Polaris, a JDBC connection is used to create metastore tables by executing SQL files. These files define types as INT and BIGINT. During the bootstrap process, when Polaris attempts to read INT columns from CockroachDB as the Integer Java datatype, it fails because CockroachDB stores these as INT8, which should be read as Long in Java.
One option would be to update sql scripts to use INT4 and INT8 instead of INT, it should work with both PostgreSQL and Cockroach DB.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Cockroach DB as Metastore [polaris]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on issue #2464:
URL: https://github.com/apache/polaris/issues/2464#issuecomment-3234897403
> The Polaris JDBC impl. still expects a 4 bytes integer. We will need to fix these as wel
precisely, have you tried this out with E2E tests (can try enabling it via overriding the profile [here](https://github.com/apache/polaris/pull/1371/files#diff-fde7952bb8b0cc8a6bc6959316008613eac974d0ef0afe0e55a51c1d010c3eb8)) if there are additional gotchas
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Cockroach DB as Metastore [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on issue #2464:
URL: https://github.com/apache/polaris/issues/2464#issuecomment-3234342530
Hi @sathesuraj, it's great idea to support Cockroach. Updating sql scripts itself only solve the part of the problem. The Polaris JDBC impl. still expects a 4 bytes integer. We will need to fix these as well. cc @singhpk234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Cockroach DB as Metastore [polaris]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 closed issue #2464: Support Cockroach DB as Metastore
URL: https://github.com/apache/polaris/issues/2464
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Cockroach DB as Metastore [polaris]
Posted by "sathesuraj (via GitHub)" <gi...@apache.org>.
sathesuraj commented on issue #2464:
URL: https://github.com/apache/polaris/issues/2464#issuecomment-3236401723
@flyrain / @singhpk234 - I haven't tried using E2E tests yet.
In CockroachDB, executing the schema SQL results in all INT types being created and stored as INT8. However, if you explicitly specify the type as INT4 in the SQL, the column is saved with the type alias as INT4, though it is still stored internally as INT8.
To make it work with CockroachDB, I explicitly specified the type as INT4 in the SQL files and executed them before running the bootstrap process. This created an alias for those columns as INT4, even though they are internally stored as INT8. This approach allows the columns to be read as Integer in Java, enabling the use of CockroachDB as a metastore for Polaris.
For example, the types were updated for columns in the "entities" table were - entity_version, type_code, sub_type_code, and grant_record_version.
No code or config change was required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Cockroach DB as Metastore [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on issue #2464:
URL: https://github.com/apache/polaris/issues/2464#issuecomment-3237974276
Hi @sathesuraj, that sounds good to me. Looking forward to your PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org