You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ilicmarkodb (via GitHub)" <gi...@apache.org> on 2026/02/05 11:35:52 UTC
[PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
ilicmarkodb opened a new pull request, #54159:
URL: https://github.com/apache/spark/pull/54159
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
8. If you want to add or modify an error type or message, please read the guideline first in
'common/utils/src/main/resources/error/README.md'.
-->
### What changes were proposed in this pull request?
Fixed `SHOW CREATE TABLE` for tables / views to correctly print `DEFAULT COLLATION collationName`.
### Why are the changes needed?
Bug fix.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
`show-create-table.sql` golden file.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-3909558418
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-3853053889
## JIRA Issue Information
=== Bug SPARK-55372 ===
Summary: Fix create table with default collation
Assignee: None
Status: Open
Affected: ["4.1.2"]
---
*This comment was automatically generated by GitHub Actions*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-4449397643
@pan3793 feel free to open a backport PR as you need :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-4412453446
@cloud-fan do you plan to backport this to 4.1? if so, I will close https://github.com/apache/spark/pull/55780
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-3909563799
@ilicmarkodb can you open a backport PR for branch 4.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "ilicmarkodb (via GitHub)" <gi...@apache.org>.
ilicmarkodb commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-3856485266
@dongjoon-hyun can you please take a look?
error doesn't look related to my change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #54159: [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation
URL: https://github.com/apache/spark/pull/54159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "ilicmarkodb (via GitHub)" <gi...@apache.org>.
ilicmarkodb commented on PR #54159:
URL: https://github.com/apache/spark/pull/54159#issuecomment-3879866623
@cloud-fan can you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation [spark]
Posted by "ilicmarkodb (via GitHub)" <gi...@apache.org>.
ilicmarkodb commented on code in PR #54159:
URL: https://github.com/apache/spark/pull/54159#discussion_r2792635562
##########
sql/connect/common/src/main/scala/org/apache/spark/sql/connect/common/DataTypeProtoConverter.scala:
##########
@@ -214,13 +214,15 @@ object DataTypeProtoConverter {
// StringType must be matched after CharType and VarcharType
case s: StringType =>
+ val stringBuilder = proto.DataType.String.newBuilder()
+ // Send collation only for explicit collations (including explicit UTF8_BINARY).
+ // Default STRING (case object) has no explicit collation and should omit it.
+ if (!s.eq(StringType)) {
+ stringBuilder.setCollation(CollationFactory.fetchCollation(s.collationId).collationName)
+ }
proto.DataType
.newBuilder()
- .setString(
- proto.DataType.String
- .newBuilder()
- .setCollation(CollationFactory.fetchCollation(s.collationId).collationName)
- .build())
+ .setString(stringBuilder.build())
Review Comment:
Without this change, a user who uses JDBC, for example, and doesn't care about collations would suddenly get `COLLATE UTF8_BINARY`, as in this test case:
https://github.com/apache/spark/blob/9c45ce4f1220650414eaf737c02780bcd6f93913/sql/connect/client/jdbc/src/test/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectJdbcDataTypeSuite.scala#L216
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org