You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by dt...@apache.org on 2026/10/20 20:16:08 UTC
(spark) branch branch-4.x updated: [SPARK-60244][SQL] Align XML CHAR/VARCHAR map key error message with JSON
This is an automated email from the ASF dual-hosted git repository.
dtenedor pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 5c0d3b42bc9a [SPARK-60244][SQL] Align XML CHAR/VARCHAR map key error message with JSON
5c0d3b42bc9a is described below
commit 5c0d3b42bc9aa80d424c171b9dad54e3ec74a648
Author: Yahya Kisana <yk...@protonmail.com>
AuthorDate: Tue Oct 20 13:15:50 2026 -0700
[SPARK-60244][SQL] Align XML CHAR/VARCHAR map key error message with JSON
### What changes were proposed in this pull request?
Follow-up to SPARK-60108 (#59325), which added `UNSUPPORTED_JSON_CHAR_VARCHAR_MAP_KEY`. This aligns the parallel XML condition from SPARK-60103 with it. Message and formatting only; XML parsing behavior is unchanged.
1. Reword `UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY` to match the JSON message:
- Before: "XML map keys of CHAR or VARCHAR cannot be padded or trimmed. The key `<key>` is not valid for type `<dataType>`."
- After: "The XML name `<key>` is not a valid key for the map key type `<dataType>`. A CHAR key must match the declared length exactly and a VARCHAR key must not exceed it; keys are never padded or trimmed. Use a STRING map key to accept any name."
2. Cap the echoed key at 128 characters plus `...` for XML too, as JSON already does. The cap moves from `unsupportedJsonCharVarcharMapKey` into a private helper, `toSQLCharVarcharMapKey`, in `QueryExecutionErrors`, and both builders use it. JSON output is unchanged.
The message parameters (`key`, `dataType`) and SQLSTATE `0A000` are unchanged.
### Why are the changes needed?
The two conditions report the same mistake for different formats but read differently. The XML one gives no hint about the length rule or the STRING workaround. It also echoes the key in full, while XML names (element names, prefixed attribute names, `valueTag`) are user data that can be very long. This was raised in the #59325 review as a follow-up.
### Does this PR introduce _any_ user-facing change?
Only the error message text for `UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY`, which is unreleased (added in SPARK-60103 for Spark 4.4). No behavior change.
```sql
SET spark.sql.charVarchar.standardSemantics.enabled=true;
SELECT from_xml('<ROW><m><abc>1</abc></m></ROW>', 'm MAP<CHAR(2), INT>', map('mode', 'FAILFAST'));
```
The cause of `MALFORMED_RECORD_IN_PARSING`, before:
```
[UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY] XML map keys of CHAR or VARCHAR cannot be padded or trimmed.
The key 'abc' is not valid for type "CHAR(2)". SQLSTATE: 0A000
```
After:
```
[UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY] The XML name 'abc' is not a valid key for the map key type "CHAR(2)".
A CHAR key must match the declared length exactly and a VARCHAR key must not exceed it; keys are never padded or trimmed. Use a STRING map key to accept any name. SQLSTATE: 0A000
```
### How was this patch tested?
Added a case to `BasicCharVarcharTestSuite` (mirroring the existing JSON cap test): a 200-character element name into `MAP<CHAR(2), INT>` under `FAILFAST` reports `key` as 128 `a`s followed by `...`. Existing XML and JSON map-key assertions still pass.
```
build/sbt 'core/testOnly *SparkThrowableSuite' 'sql/testOnly *BasicCharVarcharTestSuite'
```
### Was this patch authored or co-authored using generative AI tooling?
Generated by: Claude Opus 5.5
Fix written by and all code manually reviewed by ykisana.
Tests, PR description, and comments by Claude.
Closes #59516 from ykisana/SPARK-60244.
Authored-by: Yahya Kisana <yk...@protonmail.com>
Signed-off-by: Daniel Tenedorio <da...@databricks.com>
(cherry picked from commit 40bba48dce561d0e88ae232c096940c2c4f6b5cd)
Signed-off-by: Daniel Tenedorio <da...@databricks.com>
---
common/utils/src/main/resources/error/error-conditions.json | 4 ++--
.../org/apache/spark/sql/errors/QueryExecutionErrors.scala | 12 +++---------
.../scala/org/apache/spark/sql/CharVarcharTestSuite.scala | 11 +++++++++++
3 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/common/utils/src/main/resources/error/error-conditions.json b/common/utils/src/main/resources/error/error-conditions.json
index e0bac4f15615..9f810bd29878 100644
--- a/common/utils/src/main/resources/error/error-conditions.json
+++ b/common/utils/src/main/resources/error/error-conditions.json
@@ -9934,8 +9934,8 @@
},
"UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY" : {
"message" : [
- "XML map keys of CHAR or VARCHAR cannot be padded or trimmed.",
- "The key <key> is not valid for type <dataType>."
+ "The XML name <key> is not a valid key for the map key type <dataType>.",
+ "A CHAR key must match the declared length exactly and a VARCHAR key must not exceed it; keys are never padded or trimmed. Use a STRING map key to accept any name."
],
"sqlState" : "0A000"
},
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
index 01727a9b84b7..0e45ba4ec191 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
@@ -2741,14 +2741,7 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
// surfaces it).
def unsupportedJsonCharVarcharMapKey(
key: UTF8String, dataType: DataType): SparkRuntimeException = {
- // The key is a raw JSON object name, which in map-shaped data is often user data (ids,
- // emails) and can run up to Jackson's 50,000-character name limit, so cap what we echo.
- val maxKeyChars = 128
- val displayKey = if (key.numChars() > maxKeyChars) {
- UTF8String.concat(key.substring(0, maxKeyChars), UTF8String.fromString("..."))
- } else {
- key
- }
+ val displayKey = CharVarcharUtils.truncateMapKeyForError(key)
new SparkRuntimeException(
errorClass = "UNSUPPORTED_JSON_CHAR_VARCHAR_MAP_KEY",
messageParameters = Map(
@@ -2763,10 +2756,11 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
*/
def unsupportedXmlCharVarcharMapKey(
key: UTF8String, dataType: DataType): SparkRuntimeException = {
+ val displayKey = CharVarcharUtils.truncateMapKeyForError(key)
new SparkRuntimeException(
errorClass = "UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY",
messageParameters = Map(
- "key" -> toSQLValue(key, StringType),
+ "key" -> toSQLValue(displayKey, StringType),
"dataType" -> toSQLType(dataType)))
}
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
index 0accd2de4e7b..a2eba0a90737 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
@@ -3204,6 +3204,17 @@ class BasicCharVarcharTestSuite extends SharedSparkSession {
key = "abcd",
dataTypeSql = "VARCHAR(3)")
+ // A name longer than the 128-character echo cap is truncated with a trailing "..." in the
+ // error, as for JSON object names.
+ val longName = "a" * 200
+ assertUnsupportedXmlMapKey(
+ s"""SELECT from_xml(
+ | '<ROW><m><$longName>1</$longName></m></ROW>',
+ | 'm MAP<CHAR(2), INT>',
+ | map('mode', 'FAILFAST'))""".stripMargin,
+ key = "a" * 128 + "...",
+ dataTypeSql = "CHAR(2)")
+
checkAnswer(
sql("""SELECT from_xml(
| '<ROW><m a="1"><bc>2</bc></m></ROW>',
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org