You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ur...@apache.org on 2026/09/22 19:58:57 UTC
(spark) branch master updated: [SPARK-59633][SQL] Make unary trim collation-aware for case-insensitive ICU collations
This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 552a0477189a [SPARK-59633][SQL] Make unary trim collation-aware for case-insensitive ICU collations
552a0477189a is described below
commit 552a0477189ac495882074ef8f97daf1a19f8d5a
Author: Menelaos Karavelas <63...@users.noreply.github.com>
AuthorDate: Tue Sep 22 19:58:42 2026 +0000
[SPARK-59633][SQL] Make unary trim collation-aware for case-insensitive ICU collations
### What changes were proposed in this pull request?
This PR makes unary `trim`, `ltrim`, and `rtrim` use collation-aware matching for
case-insensitive ICU collations.
The expressions now pass their collation ID to `CollationSupport`. Interpreted execution
dispatches affected collations to the existing ICU implementation with an ASCII-space trim
string. Code generation selects either the ICU or binary implementation while generating code,
avoiding per-row collation dispatch in generated code.
Binary, lowercase, and case-sensitive ICU collations continue to use the existing binary
implementation.
### Why are the changes needed?
Unary trimming currently removes only literal ASCII space, regardless of the input collation.
The explicit two-argument forms use ICU matching for ICU-collated operands. As a result, the
forms disagree under case-insensitive ICU collations, where Unicode space separators such as
NBSP compare as equivalent to ASCII space.
For example, these expressions currently produce different results:
```sql
WITH input AS (SELECT concat(chr(160), 'abc', chr(160)) AS value)
SELECT trim(value COLLATE UNICODE_CI),
trim(' ' COLLATE UNICODE_CI, value COLLATE UNICODE_CI)
FROM input;
```
The unary form preserves both NBSP characters while the explicit form returns `abc`. Both forms
should apply the same default trim string with the same collation semantics.
### Does this PR introduce _any_ user-facing change?
Yes. Unary `trim`, `ltrim`, and `rtrim` now remove characters that compare as equivalent to the
default ASCII-space trim string under case-insensitive ICU collations, matching their explicit
two-argument forms. Behavior is unchanged for other collations.
### How was this patch tested?
Added tests covering:
- Unary and explicit trim equivalence for every Unicode space-separator (`Zs`) character.
- `UNICODE_CI`, `UNICODE_CI_AI`, and their `_RTRIM` variants.
- Unchanged behavior for binary, lowercase, and case-sensitive ICU collations.
- Negative cases for tab and zero-width space.
- Interpreted and code-generated Catalyst execution.
Ran locally:
```text
./build/sbt 'unsafe/testOnly org.apache.spark.unsafe.types.CollationSupportSuite -- *testDefaultStringTrimsUseCollation*'
./build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.expressions.StringExpressionsSuite -- -z "default trim uses collation-aware space matching"'
```
Both passed. `CollationSupportSuite#testDefaultStringTrimsUseCollation` is a JUnit test, so it is
selected with a method glob rather than ScalaTest `-z`. The Catalyst test
`default trim uses collation-aware space matching` covers interpreted and generated code.
The fork's Build and test and Report test results workflows are enabled for CI validation.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor Agent (Auto, 2026-09-18)
Closes #58908 from mkaravel/fix-unary-trim-icu-collations.
Authored-by: Menelaos Karavelas <63...@users.noreply.github.com>
Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
---
.../spark/sql/catalyst/util/CollationSupport.java | 68 ++++++++++++--
.../spark/unsafe/types/CollationSupportSuite.java | 104 +++++++++++++++++++--
.../catalyst/expressions/stringExpressions.scala | 12 +--
.../expressions/StringExpressionsSuite.scala | 42 +++++++++
.../collations-padding-trim.sql.out | 71 ++++++++++++++
.../sql-tests/inputs/collations-padding-trim.sql | 22 +++++
.../results/collations-padding-trim.sql.out | 64 +++++++++++++
7 files changed, 364 insertions(+), 19 deletions(-)
diff --git a/common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java b/common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java
index f3f3fe3449a9..bcb4e479a0ff 100644
--- a/common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java
+++ b/common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java
@@ -30,6 +30,20 @@ import java.util.regex.Pattern;
*/
public final class CollationSupport {
+ /**
+ * Whether unary trim should use collation-aware (ICU) matching for the default ASCII-space
+ * trim string, rather than the binary implementation.
+ *
+ * Only case-insensitive ICU collations (CI, CI_AI) qualify: they compare at primary/secondary
+ * ICU strength, at which UCA treats many Unicode space separators (Zs, e.g. NBSP U+00A0) as
+ * equal to U+0020. Case-sensitive ICU collations stay binary because tertiary strength
+ * distinguishes those characters; non-ICU collations (UTF8_BINARY, UTF8_LCASE) are already
+ * excluded by isCaseInsensitive; and CS_AI is not a valid trim input collation.
+ */
+ private static boolean useCollationAwareDefaultTrim(final int collationId) {
+ return CollationFactory.isCaseInsensitive(collationId);
+ }
+
/**
* Collation-aware string expressions.
*/
@@ -503,8 +517,10 @@ public final class CollationSupport {
}
public static class StringTrim {
- public static UTF8String exec(final UTF8String srcString) {
- return execBinary(srcString);
+ public static UTF8String exec(final UTF8String srcString, final int collationId) {
+ return useCollationAwareDefaultTrim(collationId) ?
+ exec(srcString, UTF8String.SPACE_UTF8, collationId) :
+ execBinary(srcString);
}
public static UTF8String exec(
final UTF8String srcString,
@@ -527,6 +543,13 @@ public final class CollationSupport {
public static String genCode(final String srcString) {
return String.format("CollationSupport.StringTrim.execBinary(%s)", srcString);
}
+ public static String genCode(final String srcString, final int collationId) {
+ if (useCollationAwareDefaultTrim(collationId)) {
+ return String.format(
+ "CollationSupport.StringTrim.execICU(%s, %d)", srcString, collationId);
+ }
+ return genCode(srcString);
+ }
public static String genCode(
final String srcString,
final String trimString,
@@ -553,6 +576,11 @@ public final class CollationSupport {
final int collationId) {
return CollationAwareUTF8String.lowercaseTrim(srcString, trimString, collationId);
}
+ public static UTF8String execICU(
+ final UTF8String srcString,
+ final int collationId) {
+ return execICU(srcString, UTF8String.SPACE_UTF8, collationId);
+ }
public static UTF8String execICU(
final UTF8String srcString,
final UTF8String trimString,
@@ -568,8 +596,10 @@ public final class CollationSupport {
}
public static class StringTrimLeft {
- public static UTF8String exec(final UTF8String srcString) {
- return execBinary(srcString);
+ public static UTF8String exec(final UTF8String srcString, final int collationId) {
+ return useCollationAwareDefaultTrim(collationId) ?
+ exec(srcString, UTF8String.SPACE_UTF8, collationId) :
+ execBinary(srcString);
}
public static UTF8String exec(
final UTF8String srcString,
@@ -589,6 +619,13 @@ public final class CollationSupport {
public static String genCode(final String srcString) {
return String.format("CollationSupport.StringTrimLeft.execBinary(%s)", srcString);
}
+ public static String genCode(final String srcString, final int collationId) {
+ if (useCollationAwareDefaultTrim(collationId)) {
+ return String.format(
+ "CollationSupport.StringTrimLeft.execICU(%s, %d)", srcString, collationId);
+ }
+ return genCode(srcString);
+ }
public static String genCode(
final String srcString,
final String trimString,
@@ -613,6 +650,11 @@ public final class CollationSupport {
final UTF8String trimString) {
return CollationAwareUTF8String.lowercaseTrimLeft(srcString, trimString);
}
+ public static UTF8String execICU(
+ final UTF8String srcString,
+ final int collationId) {
+ return execICU(srcString, UTF8String.SPACE_UTF8, collationId);
+ }
public static UTF8String execICU(
final UTF8String srcString,
final UTF8String trimString,
@@ -622,8 +664,10 @@ public final class CollationSupport {
}
public static class StringTrimRight {
- public static UTF8String exec(final UTF8String srcString) {
- return execBinary(srcString);
+ public static UTF8String exec(final UTF8String srcString, final int collationId) {
+ return useCollationAwareDefaultTrim(collationId) ?
+ exec(srcString, UTF8String.SPACE_UTF8, collationId) :
+ execBinary(srcString);
}
public static UTF8String exec(
final UTF8String srcString,
@@ -646,6 +690,13 @@ public final class CollationSupport {
public static String genCode(final String srcString) {
return String.format("CollationSupport.StringTrimRight.execBinary(%s)", srcString);
}
+ public static String genCode(final String srcString, final int collationId) {
+ if (useCollationAwareDefaultTrim(collationId)) {
+ return String.format(
+ "CollationSupport.StringTrimRight.execICU(%s, %d)", srcString, collationId);
+ }
+ return genCode(srcString);
+ }
public static String genCode(
final String srcString,
final String trimString,
@@ -671,6 +722,11 @@ public final class CollationSupport {
final int collationId) {
return CollationAwareUTF8String.lowercaseTrimRight(srcString, trimString, collationId);
}
+ public static UTF8String execICU(
+ final UTF8String srcString,
+ final int collationId) {
+ return execICU(srcString, UTF8String.SPACE_UTF8, collationId);
+ }
public static UTF8String execICU(
final UTF8String srcString,
final UTF8String trimString,
diff --git a/common/unsafe/src/test/java/org/apache/spark/unsafe/types/CollationSupportSuite.java b/common/unsafe/src/test/java/org/apache/spark/unsafe/types/CollationSupportSuite.java
index 7f3c1a925e02..edd24671f1dc 100644
--- a/common/unsafe/src/test/java/org/apache/spark/unsafe/types/CollationSupportSuite.java
+++ b/common/unsafe/src/test/java/org/apache/spark/unsafe/types/CollationSupportSuite.java
@@ -2757,11 +2757,11 @@ public class CollationSupportSuite {
if (trimString == null) {
// Trim string is ASCII space.
- result = CollationSupport.StringTrim.exec(src);
- UTF8String trimLeft = CollationSupport.StringTrimLeft.exec(src);
- resultTrimLeftRight = CollationSupport.StringTrimRight.exec(trimLeft);
- UTF8String trimRight = CollationSupport.StringTrimRight.exec(src);
- resultTrimRightLeft = CollationSupport.StringTrimLeft.exec(trimRight);
+ result = CollationSupport.StringTrim.exec(src, collationId);
+ UTF8String trimLeft = CollationSupport.StringTrimLeft.exec(src, collationId);
+ resultTrimLeftRight = CollationSupport.StringTrimRight.exec(trimLeft, collationId);
+ UTF8String trimRight = CollationSupport.StringTrimRight.exec(src, collationId);
+ resultTrimRightLeft = CollationSupport.StringTrimLeft.exec(trimRight, collationId);
} else {
// Trim string is specified.
result = CollationSupport.StringTrim.exec(src, trim, collationId);
@@ -2778,6 +2778,96 @@ public class CollationSupportSuite {
assertEquals(resultTrimRightLeft, result);
}
+ private void assertDefaultStringTrims(
+ String collationName,
+ String sourceString,
+ String expectedLeft,
+ String expectedRight,
+ String expectedBoth) throws SparkException {
+ int collationId = CollationFactory.collationNameToId(collationName);
+ UTF8String source = UTF8String.fromString(sourceString);
+ UTF8String defaultTrimString = UTF8String.fromString(" ");
+
+ UTF8String trimLeft = CollationSupport.StringTrimLeft.exec(source, collationId);
+ UTF8String trimRight = CollationSupport.StringTrimRight.exec(source, collationId);
+ UTF8String trimBoth = CollationSupport.StringTrim.exec(source, collationId);
+
+ assertEquals(UTF8String.fromString(expectedLeft), trimLeft);
+ assertEquals(UTF8String.fromString(expectedRight), trimRight);
+ assertEquals(UTF8String.fromString(expectedBoth), trimBoth);
+ assertEquals(
+ CollationSupport.StringTrimLeft.exec(source, defaultTrimString, collationId), trimLeft);
+ assertEquals(
+ CollationSupport.StringTrimRight.exec(source, defaultTrimString, collationId), trimRight);
+ assertEquals(
+ CollationSupport.StringTrim.exec(source, defaultTrimString, collationId), trimBoth);
+ }
+
+ @Test
+ public void testDefaultStringTrimsUseCollation() throws SparkException {
+ String[] spaceSeparators = {
+ "\u00A0", "\u1680", "\u2000", "\u2001", "\u2002", "\u2003", "\u2004", "\u2005",
+ "\u2006", "\u2007", "\u2008", "\u2009", "\u200A", "\u202F", "\u205F", "\u3000"
+ };
+ String[] unaffectedCollations = {
+ UTF8_BINARY,
+ "UTF8_BINARY_RTRIM",
+ UTF8_LCASE,
+ "UTF8_LCASE_RTRIM",
+ UNICODE,
+ "UNICODE_RTRIM"
+ };
+ String[] affectedCollations = {
+ UNICODE_CI,
+ "UNICODE_CI_RTRIM",
+ "UNICODE_CI_AI",
+ "UNICODE_CI_AI_RTRIM"
+ };
+
+ for (String collation : unaffectedCollations) {
+ int collationId = CollationFactory.collationNameToId(collation);
+ assertEquals(
+ "CollationSupport.StringTrim.execBinary(source)",
+ CollationSupport.StringTrim.genCode("source", collationId));
+ assertEquals(
+ "CollationSupport.StringTrimLeft.execBinary(source)",
+ CollationSupport.StringTrimLeft.genCode("source", collationId));
+ assertEquals(
+ "CollationSupport.StringTrimRight.execBinary(source)",
+ CollationSupport.StringTrimRight.genCode("source", collationId));
+ }
+ for (String collation : affectedCollations) {
+ int collationId = CollationFactory.collationNameToId(collation);
+ assertEquals(
+ String.format("CollationSupport.StringTrim.execICU(source, %d)", collationId),
+ CollationSupport.StringTrim.genCode("source", collationId));
+ assertEquals(
+ String.format("CollationSupport.StringTrimLeft.execICU(source, %d)", collationId),
+ CollationSupport.StringTrimLeft.genCode("source", collationId));
+ assertEquals(
+ String.format("CollationSupport.StringTrimRight.execICU(source, %d)", collationId),
+ CollationSupport.StringTrimRight.genCode("source", collationId));
+ }
+
+ for (String separator : spaceSeparators) {
+ String source = separator + "abc" + separator;
+ for (String collation : unaffectedCollations) {
+ assertDefaultStringTrims(collation, source, source, source, source);
+ }
+ for (String collation : affectedCollations) {
+ assertDefaultStringTrims(
+ collation, source, "abc" + separator, separator + "abc", "abc");
+ }
+ }
+
+ for (String collation : affectedCollations) {
+ assertDefaultStringTrims(collation, " abc ", "abc ", " abc", "abc");
+ assertDefaultStringTrims(collation, "\tabc\t", "\tabc\t", "\tabc\t", "\tabc\t");
+ assertDefaultStringTrims(
+ collation, "\u200Babc\u200B", "\u200Babc\u200B", "\u200Babc\u200B", "\u200Babc\u200B");
+ }
+ }
+
@Test
public void testStringTrim() throws SparkException {
// Basic tests.
@@ -3070,7 +3160,7 @@ public class CollationSupportSuite {
if (trimString == null) {
// Trim string is ASCII space.
- result = CollationSupport.StringTrimLeft.exec(src);
+ result = CollationSupport.StringTrimLeft.exec(src, collationId);
} else {
// Trim string is specified.
result = CollationSupport.StringTrimLeft.exec(src, trim, collationId);
@@ -3370,7 +3460,7 @@ public class CollationSupportSuite {
if (trimString == null) {
// Trim string is ASCII space.
- result = CollationSupport.StringTrimRight.exec(src);
+ result = CollationSupport.StringTrimRight.exec(src, collationId);
} else {
// Trim string is specified.
result = CollationSupport.StringTrimRight.exec(src, trim, collationId);
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
index b918591a43ac..d05c3ab65215 100755
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
@@ -1490,11 +1490,11 @@ trait String2TrimExpression extends Expression with ImplicitCastInputTypes {
if (evals.length == 1) {
val stringTrimCode: String = this match {
case _: StringTrim =>
- CollationSupport.StringTrim.genCode(srcString.value)
+ CollationSupport.StringTrim.genCode(srcString.value, collationId)
case _: StringTrimLeft =>
- CollationSupport.StringTrimLeft.genCode(srcString.value)
+ CollationSupport.StringTrimLeft.genCode(srcString.value, collationId)
case _: StringTrimRight =>
- CollationSupport.StringTrimRight.genCode(srcString.value)
+ CollationSupport.StringTrimRight.genCode(srcString.value, collationId)
}
ev.copy(code = code"""
|${srcString.code}
@@ -1622,7 +1622,7 @@ case class StringTrim(srcStr: Expression, trimStr: Option[Expression] = None)
override protected def direction: String = "BOTH"
override def doEval(srcString: UTF8String): UTF8String =
- CollationSupport.StringTrim.exec(srcString)
+ CollationSupport.StringTrim.exec(srcString, collationId)
override def doEval(srcString: UTF8String, trimString: UTF8String): UTF8String =
CollationSupport.StringTrim.exec(srcString, trimString, collationId)
@@ -1739,7 +1739,7 @@ case class StringTrimLeft(srcStr: Expression, trimStr: Option[Expression] = None
override protected def direction: String = "LEADING"
override def doEval(srcString: UTF8String): UTF8String =
- CollationSupport.StringTrimLeft.exec(srcString)
+ CollationSupport.StringTrimLeft.exec(srcString, collationId)
override def doEval(srcString: UTF8String, trimString: UTF8String): UTF8String =
CollationSupport.StringTrimLeft.exec(srcString, trimString, collationId)
@@ -1807,7 +1807,7 @@ case class StringTrimRight(srcStr: Expression, trimStr: Option[Expression] = Non
override protected def direction: String = "TRAILING"
override def doEval(srcString: UTF8String): UTF8String =
- CollationSupport.StringTrimRight.exec(srcString)
+ CollationSupport.StringTrimRight.exec(srcString, collationId)
override def doEval(srcString: UTF8String, trimString: UTF8String): UTF8String =
CollationSupport.StringTrimRight.exec(srcString, trimString, collationId)
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
index 4ff0a9788d99..83dbecf13228 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
@@ -28,6 +28,7 @@ import org.apache.spark.sql.catalyst.dsl.expressions._
import org.apache.spark.sql.catalyst.expressions.Cast._
import org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection
import org.apache.spark.sql.catalyst.util.CharsetProvider
+import org.apache.spark.sql.catalyst.util.CollationFactory
import org.apache.spark.sql.errors.QueryExecutionErrors.toSQLId
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.internal.types.StringTypeWithCollation
@@ -1060,6 +1061,47 @@ class StringExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
checkEvaluation(StringTrim(Literal("xxxbarxxx"), Literal("x")), "bar")
}
+ test("default trim uses collation-aware space matching") {
+ val source = "\u00A0abc\u00A0"
+
+ // Case-insensitive ICU collations treat NBSP as equal to ASCII space, so the default
+ // (unary) trim removes it, matching the explicit two-argument form with a space trim string.
+ for (collationName <- Seq("UNICODE_CI", "UNICODE_CI_AI", "en_CI")) {
+ val collationId = CollationFactory.collationNameToId(collationName)
+ val src = Literal.create(source, StringType(collationId))
+ val space = Literal.create(" ", StringType(collationId))
+
+ checkEvaluation(StringTrimLeft(src), "abc\u00A0")
+ checkEvaluation(StringTrimRight(src), "\u00A0abc")
+ checkEvaluation(StringTrim(src), "abc")
+
+ // The unary form matches the explicit two-argument form with a space trim string.
+ checkEvaluation(StringTrimLeft(src), StringTrimLeft(src, space).eval())
+ checkEvaluation(StringTrimRight(src), StringTrimRight(src, space).eval())
+ checkEvaluation(StringTrim(src), StringTrim(src, space).eval())
+
+ // Mixed padding of NBSP and ASCII space is fully trimmed.
+ val mixed = Literal.create("\u00A0 abc \u00A0", StringType(collationId))
+ checkEvaluation(StringTrim(mixed), "abc")
+
+ // A string consisting solely of space separators trims to empty.
+ val allSpaces = Literal.create("\u00A0 \u00A0", StringType(collationId))
+ checkEvaluation(StringTrimLeft(allSpaces), "")
+ checkEvaluation(StringTrimRight(allSpaces), "")
+ checkEvaluation(StringTrim(allSpaces), "")
+ }
+
+ // Case-sensitive collations keep the existing binary behavior: NBSP is preserved.
+ for (collationName <- Seq("UNICODE", "UTF8_BINARY", "UTF8_LCASE")) {
+ val collationId = CollationFactory.collationNameToId(collationName)
+ val src = Literal.create(source, StringType(collationId))
+
+ checkEvaluation(StringTrimLeft(src), source)
+ checkEvaluation(StringTrimRight(src), source)
+ checkEvaluation(StringTrim(src), source)
+ }
+ }
+
test("LTRIM") {
val s = $"a".string.at(0)
checkEvaluation(StringTrimLeft(Literal(" aa ")), "aa ", create_row(" abdef "))
diff --git a/sql/core/src/test/resources/sql-tests/analyzer-results/collations-padding-trim.sql.out b/sql/core/src/test/resources/sql-tests/analyzer-results/collations-padding-trim.sql.out
index 966641170e2d..db440f415c16 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/collations-padding-trim.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/collations-padding-trim.sql.out
@@ -683,6 +683,77 @@ Project [rtrim(cast(utf8_binary#x as string collate UTF8_LCASE), Some(collate(AB
+- Relation spark_catalog.default.t1[s#x,utf8_binary#x,utf8_lcase#x] parquet
+-- !query
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query analysis
+Project [trim(s#x, None) AS trim(s)#x, length(trim(s#x, None)) AS length(trim(s))#x, octet_length(trim(s#x, None)) AS octet_length(trim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode_ci) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select trim(' ' collate unicode_ci, s), length(trim(' ' collate unicode_ci, s)),
+ octet_length(trim(' ' collate unicode_ci, s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query analysis
+Project [trim(s#x, Some(collate( , unicode_ci))) AS TRIM(BOTH collate( , unicode_ci) FROM s)#x, length(trim(s#x, Some(collate( , unicode_ci)))) AS length(TRIM(BOTH collate( , unicode_ci) FROM s))#x, octet_length(trim(s#x, Some(collate( , unicode_ci)))) AS octet_length(TRIM(BOTH collate( , unicode_ci) FROM s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode_ci) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query analysis
+Project [ltrim(s#x, None) AS ltrim(s)#x, length(ltrim(s#x, None)) AS length(ltrim(s))#x, octet_length(ltrim(s#x, None)) AS octet_length(ltrim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode_ci) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query analysis
+Project [rtrim(s#x, None) AS rtrim(s)#x, length(rtrim(s#x, None)) AS length(rtrim(s))#x, octet_length(rtrim(s#x, None)) AS octet_length(rtrim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode_ci) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query analysis
+Project [trim(s#x, None) AS trim(s)#x, length(trim(s#x, None)) AS length(trim(s))#x, octet_length(trim(s#x, None)) AS octet_length(trim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query analysis
+Project [ltrim(s#x, None) AS ltrim(s)#x, length(ltrim(s#x, None)) AS length(ltrim(s))#x, octet_length(ltrim(s#x, None)) AS octet_length(ltrim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode) AS s#x]
+ +- OneRowRelation
+
+
+-- !query
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query analysis
+Project [rtrim(s#x, None) AS rtrim(s)#x, length(rtrim(s#x, None)) AS length(rtrim(s))#x, octet_length(rtrim(s#x, None)) AS octet_length(rtrim(s))#x]
++- SubqueryAlias __auto_generated_subquery_name
+ +- Project [collate(concat(chr(cast(160 as bigint)), abc, chr(cast(160 as bigint))), unicode) AS s#x]
+ +- OneRowRelation
+
+
-- !query
drop table t1
-- !query analysis
diff --git a/sql/core/src/test/resources/sql-tests/inputs/collations-padding-trim.sql b/sql/core/src/test/resources/sql-tests/inputs/collations-padding-trim.sql
index b86438d117d1..6785a5b4d8ea 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/collations-padding-trim.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/collations-padding-trim.sql
@@ -78,4 +78,26 @@ select RTRIM(utf8_binary collate utf8_binary_rtrim, utf8_lcase collate utf8_bina
select RTRIM('ABc', utf8_binary), RTRIM('ABc', utf8_lcase) from t1;
select RTRIM('ABc' collate utf8_lcase, utf8_binary), RTRIM('AAa' collate utf8_binary, utf8_lcase) from t1;
+-- Unary trim uses collation-aware space matching (SPARK-59633).
+-- Under UNICODE_CI, NBSP (chr(160)) compares as equal to ASCII space, so unary trim removes it
+-- and agrees with the explicit two-argument form. Under UNICODE (case-sensitive) it is preserved.
+-- The trim string is the first argument: trim(trimStr, srcStr) removes trimStr from srcStr,
+-- so the unary form below is equivalent to trim(' ', value). length/octet_length of the trimmed
+-- result disambiguate NBSP (2 UTF-8 bytes) from ASCII space, which look identical in the output.
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s);
+select trim(' ' collate unicode_ci, s), length(trim(' ' collate unicode_ci, s)),
+ octet_length(trim(' ' collate unicode_ci, s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s);
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s);
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s);
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s);
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s);
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s);
+
drop table t1;
diff --git a/sql/core/src/test/resources/sql-tests/results/collations-padding-trim.sql.out b/sql/core/src/test/resources/sql-tests/results/collations-padding-trim.sql.out
index a6377a8863e1..012211f4345a 100644
--- a/sql/core/src/test/resources/sql-tests/results/collations-padding-trim.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/collations-padding-trim.sql.out
@@ -1217,6 +1217,70 @@ kitten sitTing
İo İo
+-- !query
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query schema
+struct<trim(s):string collate UNICODE_CI,length(trim(s)):int,octet_length(trim(s)):int>
+-- !query output
+abc 3 3
+
+
+-- !query
+select trim(' ' collate unicode_ci, s), length(trim(' ' collate unicode_ci, s)),
+ octet_length(trim(' ' collate unicode_ci, s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query schema
+struct<TRIM(BOTH collate( , unicode_ci) FROM s):string collate UNICODE_CI,length(TRIM(BOTH collate( , unicode_ci) FROM s)):int,octet_length(TRIM(BOTH collate( , unicode_ci) FROM s)):int>
+-- !query output
+abc 3 3
+
+
+-- !query
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query schema
+struct<ltrim(s):string collate UNICODE_CI,length(ltrim(s)):int,octet_length(ltrim(s)):int>
+-- !query output
+abc 4 5
+
+
+-- !query
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode_ci as s)
+-- !query schema
+struct<rtrim(s):string collate UNICODE_CI,length(rtrim(s)):int,octet_length(rtrim(s)):int>
+-- !query output
+ abc 4 5
+
+
+-- !query
+select trim(s), length(trim(s)), octet_length(trim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query schema
+struct<trim(s):string collate UNICODE,length(trim(s)):int,octet_length(trim(s)):int>
+-- !query output
+ abc 5 7
+
+
+-- !query
+select ltrim(s), length(ltrim(s)), octet_length(ltrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query schema
+struct<ltrim(s):string collate UNICODE,length(ltrim(s)):int,octet_length(ltrim(s)):int>
+-- !query output
+ abc 5 7
+
+
+-- !query
+select rtrim(s), length(rtrim(s)), octet_length(rtrim(s))
+ from (select concat(chr(160), 'abc', chr(160)) collate unicode as s)
+-- !query schema
+struct<rtrim(s):string collate UNICODE,length(rtrim(s)):int,octet_length(rtrim(s)):int>
+-- !query output
+ abc 5 7
+
+
-- !query
drop table t1
-- !query schema
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org