You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/10/19 20:39:34 UTC

(spark) branch branch-4.x updated: [SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed

This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 5262b3f8d49d [SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed
5262b3f8d49d is described below

commit 5262b3f8d49d50dbd4cb27bf44e14cb4f405b2e4
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Oct 19 13:39:16 2026 -0700

    [SPARK-60344][SQL] Fix `Constraint.toDDL` to quote constraint names if needed
    
    ### What changes were proposed in this pull request?
    
    This PR aims to quote a constraint name in `Constraint.toDDL()` if needed.
    
    `BaseConstraint.toDDL()`, which `PrimaryKey`, `Unique`, `ForeignKey`, and `Check` share, already quotes the constraint columns (`NamedReference.toString`) and the referenced table of a foreign key (`Identifier.toString`) with `QuotingUtils.quoteIfNeeded`, but it writes the constraint name as it is. This PR applies `QuotingUtils.quoteIfNeeded` to the constraint name, too.
    
    ```diff
    -    return "CONSTRAINT " + name + " " + toDescription();
    +    return "CONSTRAINT " + QuotingUtils.quoteIfNeeded(name) + " " + toDescription();
    ```
    
    Like the existing quoting of the columns and the referenced table, this does not quote reserved keywords under `spark.sql.ansi.enforceReservedKeywords=true` because `Constraint.toDDL()` is a public API which does not depend on the session configuration.
    
    ### Why are the changes needed?
    
    `Constraint.toDDL()` returns the definition of a constraint in the DDL format, and `SHOW CREATE TABLE` uses it for DSv2 tables. Since the constraint name is not quoted, the output of `SHOW CREATE TABLE` cannot be run again when a constraint name is not a regular identifier. This happens not only with user-provided names but also with the names which Spark generates from the table name (`<table>_pk`, `<table>_uniq_<suffix>`, `<table>_chk_<suffix>`, and `<table>_<referenced table>_fk_<s [...]
    
    For example, consider a DSv2 table with an unnamed primary key (`InMemoryTableCatalog` of the test).
    
    ```sql
    CREATE TABLE non_part_test_catalog.ns.`tbl-1` (a INT, PRIMARY KEY (a)) USING _;
    SHOW CREATE TABLE non_part_test_catalog.ns.`tbl-1`;
    ```
    
    **BEFORE**
    
    ```
    CREATE TABLE non_part_test_catalog.ns.`tbl-1` (
      a INT NOT NULL,
      CONSTRAINT tbl-1_pk PRIMARY KEY (a) NOT ENFORCED NORELY)
    USING _
    ```
    
    Running the output again fails.
    
    ```
    [INVALID_IDENTIFIER] The unquoted identifier tbl-1_pk is invalid and must be back quoted as: `tbl-1_pk`.
    ```
    
    User-provided names fail in the same way.
    
    ```
    CONSTRAINT my pk PRIMARY KEY (a) NOT ENFORCED NORELY        -- PARSE_SYNTAX_ERROR
    CONSTRAINT uk-b UNIQUE (b) NOT ENFORCED NORELY              -- INVALID_IDENTIFIER
    CONSTRAINT fk.c FOREIGN KEY (c) REFERENCES ... (id) ...     -- PARSE_SYNTAX_ERROR
    CONSTRAINT c`1 CHECK (c IS NOT NULL) ENFORCED NORELY        -- PARSE_SYNTAX_ERROR
    ```
    
    **AFTER**
    
    ```
    CREATE TABLE non_part_test_catalog.ns.`tbl-1` (
      a INT NOT NULL,
      CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY)
    USING _
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, only for the constraint names which are not regular identifiers. `SHOW CREATE TABLE` of DSv2 tables, `Constraint.toDDL()` (and `toString()`), and the `CONSTRAINT_ALREADY_EXISTS` error message show such names with back-quotes so that the output can be run again. For example, the `CONSTRAINT_ALREADY_EXISTS` error shows the existing constraint as ``CONSTRAINT `c 1` CHECK (a > 0) ENFORCED NORELY`` instead of `CONSTRAINT c 1 CHECK (a > 0) ENFORCED NORELY`. The output for the other nam [...]
    
    This bug exists since Apache Spark 4.1.0 ([v4.1.0](https://github.com/apache/spark/releases/tag/v4.1.0), 2025-12-11) where SPARK-52142 started to show table constraints in `SHOW CREATE TABLE`.
    
    - https://github.com/apache/spark/pull/51541
    
    ### How was this patch tested?
    
    Pass the CIs with the newly added test cases.
    
    - `ConstraintSuite` checks that `toDDL()` quotes the name of every constraint type (`CHECK`, `UNIQUE`, `PRIMARY KEY`, and `FOREIGN KEY`) if needed.
    - `v2.ShowCreateTableSuite` checks that the `SHOW CREATE TABLE` output of a table, whose constraint names need quoting including a generated one, can be run again and gives the same output.
    
    Without this fix, the new `v2.ShowCreateTableSuite` test case fails like the following.
    
    ```
    - SHOW CREATE TABLE using V2 catalog V2 command: SPARK-XXXXX: show table constraints whose names need quoting *** FAILED ***
      Array(..., "CONSTRAINT tbl-1_pk PRIMARY KEY (a) NOT ENFORCED NORELY,", "CONSTRAINT uk b UNIQUE (b) NOT ENFORCED NORELY,", ...) did not equal Array(..., "CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY,", "CONSTRAINT `uk b` UNIQUE (b) NOT ENFORCED NORELY,", ...)
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    Closes #59598 from dongjoon-hyun/SPARK-60344.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit 3cefbf8afca87e2d14f7b69c60a560972a49e3d9)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../catalog/constraints/BaseConstraint.java        |  3 +-
 .../sql/connector/catalog/ConstraintSuite.scala    | 13 ++++++++
 .../command/v2/ShowCreateTableSuite.scala          | 39 ++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
index 186d2b5dedad..c6c620cc062c 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/BaseConstraint.java
@@ -19,6 +19,7 @@ package org.apache.spark.sql.connector.catalog.constraints;
 
 import java.util.StringJoiner;
 
+import org.apache.spark.sql.catalyst.util.QuotingUtils;
 import org.apache.spark.sql.connector.expressions.NamedReference;
 
 abstract class BaseConstraint implements Constraint {
@@ -65,7 +66,7 @@ abstract class BaseConstraint implements Constraint {
   public String toDDL() {
     // The validation status is not included in the DDL output as it's not part of
     // the Spark SQL syntax for constraints.
-    return "CONSTRAINT " + name + " " + toDescription();
+    return "CONSTRAINT " + QuotingUtils.quoteIfNeeded(name) + " " + toDescription();
   }
 
   public String toDescription() {
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
index 2902bef2cda0..1fed7549c27d 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/ConstraintSuite.scala
@@ -160,4 +160,17 @@ class ConstraintSuite extends SparkFunSuite {
       "NOT ENFORCED NORELY")
     assert(fk2.validationStatus() == ValidationStatus.INVALID)
   }
+
+  test("SPARK-60344: toDDL quotes the constraint name if needed") {
+    val columns = Array[NamedReference](FieldReference(Seq("a")))
+    val check = Constraint.check("c 1").predicateSql("a > 0").build()
+    assert(check.toDDL == "CONSTRAINT `c 1` CHECK (a > 0) ENFORCED NORELY")
+    val unique = Constraint.unique("uk-1", columns).build()
+    assert(unique.toDDL == "CONSTRAINT `uk-1` UNIQUE (a) ENFORCED NORELY")
+    val pk = Constraint.primaryKey("pk`1", columns).build()
+    assert(pk.toDDL == "CONSTRAINT `pk``1` PRIMARY KEY (a) ENFORCED NORELY")
+    val fk = Constraint.foreignKey(
+      "fk.1", columns, Identifier.of(Array("ns"), "t"), columns).build()
+    assert(fk.toDDL == "CONSTRAINT `fk.1` FOREIGN KEY (a) REFERENCES ns.t (a) ENFORCED NORELY")
+  }
 }
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
index bbb26f051018..961ea612e3ae 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala
@@ -276,4 +276,43 @@ class ShowCreateTableSuite extends command.ShowCreateTableSuiteBase with Command
       }
     }
   }
+
+  test("SPARK-60344: show table constraints whose names need quoting") {
+    withNamespaceAndTable("ns", "`tbl-1`", nonPartitionCatalog) { t =>
+      withNamespaceAndTable("ns", "other_table", nonPartitionCatalog) { otherTable =>
+        sql(s"CREATE TABLE $otherTable (id STRING PRIMARY KEY) $defaultUsing")
+        sql(
+          s"""
+             |CREATE TABLE $t (
+             |  a INT,
+             |  b STRING,
+             |  c STRING,
+             |  PRIMARY KEY (a),
+             |  CONSTRAINT `uk b` UNIQUE (b),
+             |  CONSTRAINT `fk.c` FOREIGN KEY (c) REFERENCES $otherTable(id) RELY,
+             |  CONSTRAINT `c``1` CHECK (c IS NOT NULL)
+             |)
+             |$defaultUsing
+        """.stripMargin)
+        val showDDL = getShowCreateDDL(t)
+        assert(showDDL === Array(
+          s"CREATE TABLE $t (",
+          "a INT NOT NULL,",
+          "b STRING COLLATE UTF8_BINARY,",
+          "c STRING COLLATE UTF8_BINARY,",
+          // The generated name of the primary key starts with the table name.
+          "CONSTRAINT `tbl-1_pk` PRIMARY KEY (a) NOT ENFORCED NORELY,",
+          "CONSTRAINT `uk b` UNIQUE (b) NOT ENFORCED NORELY,",
+          s"CONSTRAINT `fk.c` FOREIGN KEY (c) REFERENCES $otherTable (id) NOT ENFORCED RELY,",
+          "CONSTRAINT `c``1` CHECK (c IS NOT NULL) ENFORCED NORELY)",
+          defaultUsing))
+
+        // The output can be run again.
+        val ddl = sql(s"SHOW CREATE TABLE $t").head().getString(0)
+        sql(s"DROP TABLE $t")
+        sql(ddl)
+        assert(getShowCreateDDL(t) === showDDL)
+      }
+    }
+  }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org