You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/10/19 20:39:34 UTC
(spark) branch branch-4.x updated: [SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 5262b3f8d49d [SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed
5262b3f8d49d is described below
commit 5262b3f8d49d50dbd4cb27bf44e14cb4f405b2e4
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Oct 19 13:39:16 2026 -0700
[SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed
### What changes were proposed in this pull request?
This PR aims to quote a constraint name in `Constraint.toDDL()` if needed.
`BaseConstraint.toDDL()`, which `PrimaryKey`, `Unique`, `ForeignKey`, and `Check` share, already quotes the constraint columns (`NamedReference.toString`) and the referenced table of a foreign key (`Identifier.toString`) with `QuotingUtils.quoteIfNeeded`, but it writes the constraint name as it is. This PR applies `QuotingUtils.quoteIfNeeded` to the constraint name, too.
```diff
- return "CONSTRAINT " + name + " " + toDescription();
+ return "CONSTRAINT " + QuotingUtils.quoteIfNeeded(name) + " " + toDescription();
```
Like the existing quoting of the columns and the referenced table, this does not quote reserved keywords under `spark.sql.ansi.enforceReservedKeywords=true` because `Constraint.toDDL()` is a public API which does not depend on the session configuration.
### Why are the changes needed?
`Constraint.toDDL()` returns the definition of a constraint in the DDL format, and `SHOW CREATE TABLE` uses it for DSv2 tables. Since the constraint name is not quoted, the output of `SHOW CREATE TABLE` cannot be run again when a constraint name is not a regular identifier. This happens not only with user-provided names but also with the names which Spark generates from the table name (`<table>_pk`, `<table>_uniq_<suffix>`, `<table>_chk_<suffix>`, and `<table>_<referenced table>_fk_<s [...]
For example, consider a DSv2 table with an unnamed primary key (`InMemoryTableCatalog` of the test).
```sql
CREATE TABLE non_part_test_catalog.ns.`tbl-1` (a INT, PRIMARY KEY (a)) USING _;
SHOW CREATE TABLE non_part_test_catalog.ns.`tbl-1`;
```
**BEFORE**
```
CREATE TABLE non_part_test_catalog.ns.`tbl-1` (
a INT NOT NULL,
CONSTRAINT tbl-1_pk PRIMARY KEY (a) NOT ENFORCED NORELY)
USING _
```
Running the output again fails.
```
[INVALID_IDENTIFIER] The unquoted identifier tbl-1_pk is invalid and must be back quoted as: `tbl-1_pk`.
```
User-provided names fail in the same way.
```
CONSTRAINT my pk PRIMARY KEY (a) NOT ENFORCED NORELY -- PARSE_SYNTAX_ERROR
CONSTRAINT uk-b UNIQUE (b) NOT ENFORCED NORELY -- INVALID_IDENTIFIER
CONSTRAINT fk.c FOREIGN KEY (c) REFERENCES ... (id) ... -- PARSE_SYNTAX_ERROR
CONSTRAINT c`1 CHECK (c IS NOT NULL) ENFORCED NORELY -- PARSE_SYNTAX_ERROR
```
**AFTER**
```
CREATE TABLE non_part_test_catalog.ns.`tbl-1` (
a INT NOT NULL,
CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY)
USING _
```
### Does this PR introduce _any_ user-facing change?
Yes, only for the constraint names which are not regular identifiers. `SHOW CREATE TABLE` of DSv2 tables, `Constraint.toDDL()` (and `toString()`), and the `CONSTRAINT_ALREADY_EXISTS` error message show such names with back-quotes so that the output can be run again. For example, the `CONSTRAINT_ALREADY_EXISTS` error shows the existing constraint as ``CONSTRAINT `c 1` CHECK (a > 0) ENFORCED NORELY`` instead of `CONSTRAINT c 1 CHECK (a > 0) ENFORCED NORELY`. The output for the other nam [...]
This bug exists since Apache Spark 4.1.0 ([v4.1.0](https://github.com/apache/spark/releases/tag/v4.1.0), 2025-12-11) where SPARK-52142 started to show table constraints in `SHOW CREATE TABLE`.
- https://github.com/apache/spark/pull/51541
### How was this patch tested?
Pass the CIs with the newly added test cases.
- `ConstraintSuite` checks that `toDDL()` quotes the name of every constraint type (`CHECK`, `UNIQUE`, `PRIMARY KEY`, and `FOREIGN KEY`) if needed.
- `v2.ShowCreateTableSuite` checks that the `SHOW CREATE TABLE` output of a table, whose constraint names need quoting including a generated one, can be run again and gives the same output.
Without this fix, the new `v2.ShowCreateTableSuite` test case fails like the following.
```
- SHOW CREATE TABLE using V2 catalog V2 command: SPARK-XXXXX: show table constraints whose names need quoting *** FAILED ***
Array(..., "CONSTRAINT tbl-1_pk PRIMARY KEY (a) NOT ENFORCED NORELY,", "CONSTRAINT uk b UNIQUE (b) NOT ENFORCED NORELY,", ...) did not equal Array(..., "CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY,", "CONSTRAINT `uk b` UNIQUE (b) NOT ENFORCED NORELY,", ...)
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #59598 from dongjoon-hyun/SPARK-60344.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 3cefbf8afca87e2d14f7b69c60a560972a49e3d9)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../catalog/constraints/BaseConstraint.java | 3 +-
.../sql/connector/catalog/ConstraintSuite.scala | 13 ++++++++
.../command/v2/ShowCreateTableSuite.scala | 39 ++++++++++++++++++++++
3 files changed, 54 insertions(+), 1 deletion(-)
diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
index 186d2b5dedad..c6c620cc062c 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
@@ -19,6 +19,7 @@ package org.apache.spark.sql.connector.catalog.constraints;
import java.util.StringJoiner;
+import org.apache.spark.sql.catalyst.util.QuotingUtils;
import org.apache.spark.sql.connector.expressions.NamedReference;
abstract class BaseConstraint implements Constraint {
@@ -65,7 +66,7 @@ abstract class BaseConstraint implements Constraint {
public String toDDL() {
// The validation status is not included in the DDL output as it's not part of
// the Spark SQL syntax for constraints.
- return "CONSTRAINT " + name + " " + toDescription();
+ return "CONSTRAINT " + QuotingUtils.quoteIfNeeded(name) + " " + toDescription();
}
public String toDescription() {
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
index 2902bef2cda0..1fed7549c27d 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
@@ -160,4 +160,17 @@ class ConstraintSuite extends SparkFunSuite {
"NOT ENFORCED NORELY")
assert(fk2.validationStatus() == ValidationStatus.INVALID)
}
+
+ test("SPARK-60344: toDDL quotes the constraint name if needed") {
+ val columns = Array[NamedReference](FieldReference(Seq("a")))
+ val check = Constraint.check("c 1").predicateSql("a > 0").build()
+ assert(check.toDDL == "CONSTRAINT `c 1` CHECK (a > 0) ENFORCED NORELY")
+ val unique = Constraint.unique("uk-1", columns).build()
+ assert(unique.toDDL == "CONSTRAINT `uk-1` UNIQUE (a) ENFORCED NORELY")
+ val pk = Constraint.primaryKey("pk`1", columns).build()
+ assert(pk.toDDL == "CONSTRAINT `pk``1` PRIMARY KEY (a) ENFORCED NORELY")
+ val fk = Constraint.foreignKey(
+ "fk.1", columns, Identifier.of(Array("ns"), "t"), columns).build()
+ assert(fk.toDDL == "CONSTRAINT `fk.1` FOREIGN KEY (a) REFERENCES ns.t (a) ENFORCED NORELY")
+ }
}
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
index bbb26f051018..961ea612e3ae 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
@@ -276,4 +276,43 @@ class ShowCreateTableSuite extends command.ShowCreateTableSuiteBase with Command
}
}
}
+
+ test("SPARK-60344: show table constraints whose names need quoting") {
+ withNamespaceAndTable("ns", "`tbl-1`", nonPartitionCatalog) { t =>
+ withNamespaceAndTable("ns", "other_table", nonPartitionCatalog) { otherTable =>
+ sql(s"CREATE TABLE $otherTable (id STRING PRIMARY KEY) $defaultUsing")
+ sql(
+ s"""
+ |CREATE TABLE $t (
+ | a INT,
+ | b STRING,
+ | c STRING,
+ | PRIMARY KEY (a),
+ | CONSTRAINT `uk b` UNIQUE (b),
+ | CONSTRAINT `fk.c` FOREIGN KEY (c) REFERENCES $otherTable(id) RELY,
+ | CONSTRAINT `c``1` CHECK (c IS NOT NULL)
+ |)
+ |$defaultUsing
+ """.stripMargin)
+ val showDDL = getShowCreateDDL(t)
+ assert(showDDL === Array(
+ s"CREATE TABLE $t (",
+ "a INT NOT NULL,",
+ "b STRING COLLATE UTF8_BINARY,",
+ "c STRING COLLATE UTF8_BINARY,",
+ // The generated name of the primary key starts with the table name.
+ "CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY,",
+ "CONSTRAINT `uk b` UNIQUE (b) NOT ENFORCED NORELY,",
+ s"CONSTRAINT `fk.c` FOREIGN KEY (c) REFERENCES $otherTable (id) NOT ENFORCED RELY,",
+ "CONSTRAINT `c``1` CHECK (c IS NOT NULL) ENFORCED NORELY)",
+ defaultUsing))
+
+ // The output can be run again.
+ val ddl = sql(s"SHOW CREATE TABLE $t").head().getString(0)
+ sql(s"DROP TABLE $t")
+ sql(ddl)
+ assert(getShowCreateDDL(t) === showDDL)
+ }
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org