You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2026/08/22 14:28:25 UTC
(spark) branch branch-4.x updated: [SPARK-58807][SQL] Preserve CHAR/VARCHAR on language surfaces under standardSemantics
This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 85bc1023d53f [SPARK-58807][SQL] Preserve CHAR/VARCHAR on language surfaces under standardSemantics
85bc1023d53f is described below
commit 85bc1023d53f673581417906b78981d786dde082
Author: Serge Rielau <se...@rielau.com>
AuthorDate: Sat Aug 22 22:27:43 2026 +0800
[SPARK-58807][SQL] Preserve CHAR/VARCHAR on language surfaces under standardSemantics
### What changes were proposed in this pull request?
Parent epic: [SPARK-58794](https://issues.apache.org/jira/browse/SPARK-58794). Unique ticket: [SPARK-58807](https://issues.apache.org/jira/browse/SPARK-58807) (CTAS/CREATE VIEW/CTE inherit CHAR/VARCHAR). Also covers [SPARK-58808](https://issues.apache.org/jira/browse/SPARK-58808) (SQL FUNCTION assignment), [SPARK-58809](https://issues.apache.org/jira/browse/SPARK-58809) (session variables), and [SPARK-58811](https://issues.apache.org/jira/browse/SPARK-58811) (ALTER/temp/external).
#58080 (SPARK-58798 LCT) is on master. Clients (JDBC / HS2 / Connect) landed in #58132 / SPARK-58806.
Language-surface hardening under `spark.sql.charVarchar.standardSemantics.enabled`:
**Bug fixes (format write)**
- **ORC:** `CharType`/`VarcharType` extend `StringType`, so write used to stamp `spark.sql.catalyst.type=string` and file-only inference lost the constraint. Stamp `char(n)` / `varchar(n)` via `CharVarcharUtils.charVarcharTypeName`. Write stays on ORC `STRING` plus that attribute (not native ORC CHAR/VARCHAR), so ORC `maxLength` does not fight Spark store assignment. Unbounded STRING, including collated STRING, still stamps `string` (same as Avro: collation is not a file-only round-tr [...]
- **Avro:** same subclass trap on leaves, map keys, and ser/de. Stamp `spark.sql.catalyst.type` on STRING and `spark.sql.catalyst.mapKey.type` on maps. Restore only `StringType` subtypes; a non-string stamp is `IncompatibleSchemaException`. sql/core has no avro data source; ser/de is covered with `DataFileWriter`.
**Coverage**
- CTAS / CREATE VIEW (CHAR and VARCHAR) / CTE inherit CHAR/VARCHAR
- ALTER COLUMN equal-length CHAR/VARCHAR; V2 VARCHAR widen / CHAR to VARCHAR
- Session / script variables, FETCH INTO, SQL FUNCTION params and RETURNS
- EXTERNAL TABLE scan pad / overflow
- CHAR/VARCHAR vs non-string compare and COALESCE
- JSON / CSV with a user-specified CHAR/VARCHAR schema
- ORC catalog + file-only round-trip, including cross-flag (`standardSemantics` on then off, first-class types off, `preserveCharVarcharTypeInfo` only)
- Avro schema conversion, nested struct, CHAR map keys, and serializer/deserializer values
### Why are the changes needed?
Without ORC/Avro catalyst-type stamping, Spark-written files cannot re-infer CHAR/VARCHAR under the flag. Language surfaces (CTAS/VIEW/CTE/vars/functions/user schemas) need explicit coverage so schema fidelity does not regress.
### Does this PR introduce _any_ user-facing change?
Yes, when the flag is on: ORC and Avro preserve CHAR/VARCHAR logical types across write/read instead of collapsing to STRING. CTAS/VIEW/CTE/vars/functions behavior matches the foundation rules and is now tested.
### How was this patch tested?
- `BasicCharVarcharTestSuite` / language surfaces (including Avro ser/de and ORC cross-flag)
- Regenerated `charvarchar-standard-semantics.sql` goldens
Closes #58087 from srielau/SPARK-58794-surfaces.
Authored-by: Serge Rielau <se...@rielau.com>
Signed-off-by: Wenchen Fan <we...@databricks.com>
(cherry picked from commit db6fc3a7ec60834a9bf9cec34026c5e6fa06fa68)
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
.../sql/catalyst/util/SparkCharVarcharUtils.scala | 10 +
.../apache/spark/sql/avro/AvroDeserializer.scala | 8 +-
.../org/apache/spark/sql/avro/AvroSerializer.scala | 6 +-
.../apache/spark/sql/avro/SchemaConverters.scala | 63 ++-
.../sql/execution/datasources/orc/OrcUtils.scala | 16 +-
.../charvarchar-standard-semantics.sql.out | 211 +++++++++
.../inputs/charvarchar-standard-semantics.sql | 42 ++
.../results/charvarchar-standard-semantics.sql.out | 204 ++++++++
.../apache/spark/sql/CharVarcharTestSuite.scala | 520 ++++++++++++++++++++-
9 files changed, 1058 insertions(+), 22 deletions(-)
diff --git a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkCharVarcharUtils.scala b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkCharVarcharUtils.scala
index 1e3589020671..70684d392042 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkCharVarcharUtils.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkCharVarcharUtils.scala
@@ -30,6 +30,16 @@ trait SparkCharVarcharUtils {
dt.existsRecursively(f => f.isInstanceOf[CharType] || f.isInstanceOf[VarcharType])
}
+ /**
+ * Logical type name for CHAR/VARCHAR, used when stamping `spark.sql.catalyst.type` on STRING
+ * storage (ORC/Avro). None for other types, including unbounded STRING.
+ */
+ def charVarcharTypeName(dt: DataType): Option[String] = dt match {
+ case c: CharType => Some(c.typeName)
+ case v: VarcharType => Some(v.typeName)
+ case _ => None
+ }
+
/**
* Fail if the type contains CHAR/VARCHAR unless legacy-as-string or first-class CHAR/VARCHAR is
* enabled (standard semantics or preserveCharVarcharTypeInfo).
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala b/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala
index 078d80350f4d..25d0c7584908 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala
@@ -250,7 +250,7 @@ private[sql] class AvroDeserializer(
case (DOUBLE, DoubleType) => (updater, ordinal, value) =>
updater.setDouble(ordinal, value.asInstanceOf[Double])
- case (STRING, StringType) => (updater, ordinal, value) =>
+ case (STRING, _: StringType) => (updater, ordinal, value) =>
val str = value match {
case s: String => UTF8String.fromString(s)
case s: Utf8 =>
@@ -260,7 +260,7 @@ private[sql] class AvroDeserializer(
}
updater.set(ordinal, str)
- case (ENUM, StringType) => (updater, ordinal, value) =>
+ case (ENUM, _: StringType) => (updater, ordinal, value) =>
updater.set(ordinal, UTF8String.fromString(value.toString))
case (FIXED, BinaryType) => (updater, ordinal, value) =>
@@ -345,8 +345,8 @@ private[sql] class AvroDeserializer(
updater.set(ordinal, result)
- case (MAP, MapType(keyType, valueType, valueContainsNull)) if keyType == StringType =>
- val keyWriter = newWriter(SchemaBuilder.builder().stringType(), StringType,
+ case (MAP, MapType(keyType: StringType, valueType, valueContainsNull)) =>
+ val keyWriter = newWriter(SchemaBuilder.builder().stringType(), keyType,
avroPath :+ "key", catalystPath :+ "key")
val valueWriter = newWriter(avroType.getValueType, valueType,
avroPath :+ "value", catalystPath :+ "value")
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala b/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala
index d5405f69ff05..353ac2eecbf1 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala
@@ -137,7 +137,7 @@ private[sql] class AvroSerializer(
decimalConversions.toBytes(decimal.toJavaBigDecimal, avroType,
LogicalTypes.decimal(d.precision, d.scale))
- case (StringType, ENUM) =>
+ case (_: StringType, ENUM) =>
val enumSymbols: Set[String] = avroType.getEnumSymbols.asScala.toSet
(getter, ordinal) =>
val data = getter.getUTF8String(ordinal).toString
@@ -148,7 +148,7 @@ private[sql] class AvroSerializer(
}
new EnumSymbol(avroType, data)
- case (StringType, STRING) =>
+ case (_: StringType, STRING) =>
(getter, ordinal) => new Utf8(getter.getUTF8String(ordinal).getBytes)
case (BinaryType, FIXED) =>
@@ -259,7 +259,7 @@ private[sql] class AvroSerializer(
case (LongType, UNION) if nonNullUnionTypes(avroType) == Set(INT, LONG) =>
(getter, ordinal) => getter.getLong(ordinal)
- case (MapType(kt, vt, valueContainsNull), MAP) if kt == StringType =>
+ case (MapType(_: StringType, vt, valueContainsNull), MAP) =>
val valueConverter = newConverter(
vt, resolveNullableType(avroType.getValueType, valueContainsNull),
catalystPath :+ "value", avroPath :+ "value")
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala b/sql/core/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala
index c6fba163309f..9ecbe1b6a0f5 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala
@@ -32,6 +32,7 @@ import org.apache.spark.internal.Logging
import org.apache.spark.internal.LogKeys.{FIELD_NAME, FIELD_TYPE, RECURSIVE_DEPTH}
import org.apache.spark.sql.avro.AvroOptions.RECURSIVE_FIELD_MAX_DEPTH_LIMIT
import org.apache.spark.sql.catalyst.parser.CatalystSqlParser
+import org.apache.spark.sql.catalyst.util.CharVarcharUtils
import org.apache.spark.sql.types._
import org.apache.spark.sql.types.Decimal.minBytesForPrecision
@@ -95,6 +96,33 @@ object SchemaConverters extends Logging {
// The property specifies Catalyst type of the given field
private val CATALYST_TYPE_PROP_NAME = "spark.sql.catalyst.type"
+ // Avro map keys are always STRING; stamp CHAR/VARCHAR key types on the map schema.
+ private val CATALYST_MAP_KEY_TYPE_PROP_NAME = "spark.sql.catalyst.mapKey.type"
+
+ private def avroStringSchema(catalystType: StringType): Schema = {
+ val stringSchema = SchemaBuilder.builder().stringType()
+ CharVarcharUtils.charVarcharTypeName(catalystType).foreach { name =>
+ stringSchema.addProp(CATALYST_TYPE_PROP_NAME, name)
+ }
+ stringSchema
+ }
+
+ private def avroMapSchema(keyType: StringType, valueSchema: Schema): Schema = {
+ val mapSchema = SchemaBuilder.builder().map().values(valueSchema)
+ CharVarcharUtils.charVarcharTypeName(keyType).foreach { name =>
+ mapSchema.addProp(CATALYST_MAP_KEY_TYPE_PROP_NAME, name)
+ }
+ mapSchema
+ }
+
+ private def parseStampedStringType(catalystTypeAttrValue: String): StringType = {
+ CatalystSqlParser.parseDataType(catalystTypeAttrValue) match {
+ case s: StringType => s
+ case other =>
+ throw new IncompatibleSchemaException(
+ s"Avro $CATALYST_TYPE_PROP_NAME for STRING must be a STRING subtype, got $other")
+ }
+ }
private def toSqlTypeHelper(
avroSchema: Schema,
@@ -114,7 +142,14 @@ object SchemaConverters extends Logging {
}
SchemaType(catalystType, nullable = false)
}
- case STRING => SchemaType(StringType, nullable = false)
+ case STRING =>
+ val catalystTypeAttrValue = avroSchema.getProp(CATALYST_TYPE_PROP_NAME)
+ val catalystType = if (catalystTypeAttrValue == null) {
+ StringType
+ } else {
+ parseStampedStringType(catalystTypeAttrValue)
+ }
+ SchemaType(catalystType, nullable = false)
case BOOLEAN => SchemaType(BooleanType, nullable = false)
case BYTES | FIXED => avroSchema.getLogicalType match {
// For FIXED type, if the precision requires more bytes than fixed size, the logical
@@ -244,8 +279,14 @@ object SchemaConverters extends Logging {
)
null
} else {
+ val keyAttr = avroSchema.getProp(CATALYST_MAP_KEY_TYPE_PROP_NAME)
+ val keyType = if (keyAttr == null) {
+ StringType
+ } else {
+ parseStampedStringType(keyAttr)
+ }
SchemaType(
- MapType(StringType, schemaType.dataType, valueContainsNull = schemaType.nullable),
+ MapType(keyType, schemaType.dataType, valueContainsNull = schemaType.nullable),
nullable = false)
}
@@ -369,7 +410,9 @@ object SchemaConverters extends Logging {
case FloatType => builder.floatType()
case DoubleType => builder.doubleType()
- case StringType => builder.stringType()
+ // CharType/VarcharType are StringType subclasses, not the StringType singleton.
+ // Stamp spark.sql.catalyst.type so inference restores the length constraint.
+ case s: StringType => avroStringSchema(s)
case NullType => builder.nullType()
case d: DecimalType =>
val avroType = LogicalTypes.decimal(d.precision, d.scale)
@@ -385,9 +428,8 @@ object SchemaConverters extends Logging {
case ArrayType(et, containsNull) =>
builder.array()
.items(toAvroType(et, containsNull, recordName, nameSpace))
- case MapType(StringType, vt, valueContainsNull) =>
- builder.map()
- .values(toAvroType(vt, valueContainsNull, recordName, nameSpace))
+ case MapType(kt: StringType, vt, valueContainsNull) =>
+ avroMapSchema(kt, toAvroType(vt, valueContainsNull, recordName, nameSpace))
case st: StructType =>
val childNameSpace = if (nameSpace != "") s"$nameSpace.$recordName" else recordName
val fieldsAssembler = builder.record(recordName).namespace(nameSpace).fields()
@@ -489,7 +531,7 @@ object SchemaConverters extends Logging {
case LongType => builder.longType()
case FloatType => builder.floatType()
case DoubleType => builder.doubleType()
- case StringType => builder.stringType()
+ case s: StringType => avroStringSchema(s)
case NullType => builder.nullType()
case DateType => LogicalTypes.date().addToSchema(builder.intType())
case TimestampType => LogicalTypes.timestampMicros().addToSchema(builder.longType())
@@ -522,9 +564,10 @@ object SchemaConverters extends Logging {
// Make array types nullable
Schema.createUnion(nullSchema, arraySchema)
- case MapType(StringType, valueType, _) =>
- val mapSchema = builder.map()
- .values(toAvroTypeWithDefaults(valueType, recordName = recordName,
+ case MapType(kt: StringType, valueType, _) =>
+ val mapSchema = avroMapSchema(
+ kt,
+ toAvroTypeWithDefaults(valueType, recordName = recordName,
namespace = namespace, nestingLevel = nestingLevel + 1))
// Make map types nullable
Schema.createUnion(nullSchema, mapSchema)
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala
index dd51f0eaf029..f533ded6243c 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala
@@ -176,8 +176,8 @@ object OrcUtils extends Logging {
MapType(catalystKeyType, catalystValueType)
}
- // The Spark query engine has not completely supported CHAR/VARCHAR type yet, and here we
- // replace the orc CHAR/VARCHAR with STRING type.
+ // Annotate metadata for the legacy path; under charVarcharFirstClassTypes the constrained
+ // types are kept so ORC catalyst attributes / native char/varchar round-trip as first-class.
CharVarcharUtils.replaceCharVarcharWithStringInSchema(toStructType(schema))
}
@@ -469,9 +469,17 @@ object OrcUtils extends Logging {
val typeDesc = new TypeDescription(ops.orcCategory)
typeDesc.setAttribute(CATALYST_TYPE_ATTRIBUTE_NAME, dt.typeName)
Some(typeDesc)
- case _: StringType =>
+ // Write CHAR/VARCHAR as ORC STRING plus spark.sql.catalyst.type, not native
+ // ORC CHAR/VARCHAR. Native ORC maxLength would truncate/pad independently of
+ // Spark store assignment. Hive-written native CHAR still round-trips on read
+ // via toCatalystSchema. Unbounded STRING (including collated) stamps
+ // StringType.typeName ("string"), matching Avro: this PR does not round-trip
+ // collation on file-only reads.
+ case s: StringType =>
val typeDesc = new TypeDescription(TypeDescription.Category.STRING)
- typeDesc.setAttribute(CATALYST_TYPE_ATTRIBUTE_NAME, StringType.typeName)
+ typeDesc.setAttribute(
+ CATALYST_TYPE_ATTRIBUTE_NAME,
+ CharVarcharUtils.charVarcharTypeName(s).getOrElse(StringType.typeName))
Some(typeDesc)
case _ => None
}
diff --git a/sql/core/src/test/resources/sql-tests/analyzer-results/charvarchar-standard-semantics.sql.out b/sql/core/src/test/resources/sql-tests/analyzer-results/charvarchar-standard-semantics.sql.out
index 85688308aa2e..070fba1bcd94 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/charvarchar-standard-semantics.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/charvarchar-standard-semantics.sql.out
@@ -718,6 +718,80 @@ Project [cast(cast(a as char(2) collate UTF8_LCASE) as varchar(2) collate UTF8_L
+- OneRowRelation
+-- !query
+SELECT typeof(coalesce(cast('123' AS CHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS VARCHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS STRING), 1))
+-- !query analysis
+Project [typeof(coalesce(cast(cast(123 as char(3)) as bigint), cast(1 as bigint))) AS typeof(coalesce(CAST(123 AS CHAR(3)), 1))#x, typeof(coalesce(cast(cast(123 as varchar(3)) as bigint), cast(1 as bigint))) AS typeof(coalesce(CAST(123 AS VARCHAR(3)), 1))#x, typeof(coalesce(cast(cast(123 as string) as bigint), cast(1 as bigint))) AS typeof(coalesce(CAST(123 AS STRING), 1))#x]
++- OneRowRelation
+
+
+-- !query
+SELECT coalesce(cast('123' AS CHAR(3)), 1),
+ coalesce(cast('123' AS VARCHAR(3)), 1),
+ coalesce(cast('123' AS STRING), 1)
+-- !query analysis
+Project [coalesce(cast(cast(123 as char(3)) as bigint), cast(1 as bigint)) AS coalesce(CAST(123 AS CHAR(3)), 1)#xL, coalesce(cast(cast(123 as varchar(3)) as bigint), cast(1 as bigint)) AS coalesce(CAST(123 AS VARCHAR(3)), 1)#xL, coalesce(cast(cast(123 as string) as bigint), cast(1 as bigint)) AS coalesce(CAST(123 AS STRING), 1)#xL]
++- OneRowRelation
+
+
+-- !query
+SELECT cast('123' AS CHAR(3)) = 123,
+ cast('123' AS VARCHAR(3)) = 123,
+ cast('123' AS STRING) = 123
+-- !query analysis
+Project [(cast(cast(123 as char(3)) as bigint) = cast(123 as bigint)) AS (CAST(123 AS CHAR(3)) = 123)#x, (cast(cast(123 as varchar(3)) as bigint) = cast(123 as bigint)) AS (CAST(123 AS VARCHAR(3)) = 123)#x, (cast(cast(123 as string) as bigint) = cast(123 as bigint)) AS (CAST(123 AS STRING) = 123)#x]
++- OneRowRelation
+
+
+-- !query
+SELECT typeof(coalesce(cast('1.5' AS CHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS VARCHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS STRING), 1.5))
+-- !query analysis
+Project [typeof(coalesce(cast(cast(1.5 as char(3)) as double), cast(1.5 as double))) AS typeof(coalesce(CAST(1.5 AS CHAR(3)), 1.5))#x, typeof(coalesce(cast(cast(1.5 as varchar(3)) as double), cast(1.5 as double))) AS typeof(coalesce(CAST(1.5 AS VARCHAR(3)), 1.5))#x, typeof(coalesce(cast(cast(1.5 as string) as double), cast(1.5 as double))) AS typeof(coalesce(CAST(1.5 AS STRING), 1.5))#x]
++- OneRowRelation
+
+
+-- !query
+SELECT cast('2020-01-02' AS CHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS VARCHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS STRING) = date'2020-01-02'
+-- !query analysis
+[Analyzer test output redacted due to nondeterminism]
+
+
+-- !query
+SELECT cast('true' AS CHAR(4)) = true,
+ cast('true' AS VARCHAR(4)) = true,
+ cast('true' AS STRING) = true
+-- !query analysis
+Project [(cast(cast(true as char(4)) as boolean) = true) AS (CAST(true AS CHAR(4)) = true)#x, (cast(cast(true as varchar(4)) as boolean) = true) AS (CAST(true AS VARCHAR(4)) = true)#x, (cast(cast(true as string) as boolean) = true) AS (CAST(true AS STRING) = true)#x]
++- OneRowRelation
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS CHAR(4)), true))
+-- !query analysis
+Project [typeof(coalesce(cast(cast(true as char(4)) as boolean), true)) AS typeof(coalesce(CAST(true AS CHAR(4)), true))#x]
++- OneRowRelation
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS VARCHAR(4)), true))
+-- !query analysis
+Project [typeof(coalesce(cast(cast(true as varchar(4)) as boolean), true)) AS typeof(coalesce(CAST(true AS VARCHAR(4)), true))#x]
++- OneRowRelation
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS STRING), true))
+-- !query analysis
+Project [typeof(coalesce(cast(cast(true as string) as boolean), true)) AS typeof(coalesce(CAST(true AS STRING), true))#x]
++- OneRowRelation
+
+
-- !query
SELECT typeof(array(cast('a' AS CHAR(2)), cast('bb' AS CHAR(3))))
-- !query analysis
@@ -780,6 +854,143 @@ Project [length(cast(c#x as string)) AS length(c)#x, length(cast(v#x as string))
+- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+-- !query
+CREATE TABLE char_varchar_std_ctas USING parquet AS SELECT c, v FROM char_varchar_std
+-- !query analysis
+CreateDataSourceTableAsSelectCommand `spark_catalog`.`default`.`char_varchar_std_ctas`, ErrorIfExists, [c, v]
+ +- Project [c#x, v#x]
+ +- SubqueryAlias spark_catalog.default.char_varchar_std
+ +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+
+
+-- !query
+SELECT typeof(c), typeof(v) FROM char_varchar_std_ctas
+-- !query analysis
+Project [typeof(c#x) AS typeof(c)#x, typeof(v#x) AS typeof(v)#x]
++- SubqueryAlias spark_catalog.default.char_varchar_std_ctas
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std_ctas[c#x,v#x] parquet
+
+
+-- !query
+CREATE VIEW char_varchar_std_view AS SELECT c FROM char_varchar_std
+-- !query analysis
+CreateViewCommand `spark_catalog`.`default`.`char_varchar_std_view`, SELECT c FROM char_varchar_std, false, false, PersistedView, COMPENSATION, true
+ +- Project [c#x]
+ +- SubqueryAlias spark_catalog.default.char_varchar_std
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+
+
+-- !query
+SELECT typeof(c) FROM char_varchar_std_view
+-- !query analysis
+Project [typeof(c#x) AS typeof(c)#x]
++- SubqueryAlias spark_catalog.default.char_varchar_std_view
+ +- View (`spark_catalog`.`default`.`char_varchar_std_view`, [c#x])
+ +- Project [cast(c#x as char(5)) AS c#x]
+ +- Project [c#x]
+ +- SubqueryAlias spark_catalog.default.char_varchar_std
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+
+
+-- !query
+CREATE VIEW char_varchar_std_view_v AS SELECT v FROM char_varchar_std
+-- !query analysis
+CreateViewCommand `spark_catalog`.`default`.`char_varchar_std_view_v`, SELECT v FROM char_varchar_std, false, false, PersistedView, COMPENSATION, true
+ +- Project [v#x]
+ +- SubqueryAlias spark_catalog.default.char_varchar_std
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+
+
+-- !query
+SELECT typeof(v) FROM char_varchar_std_view_v
+-- !query analysis
+Project [typeof(v#x) AS typeof(v)#x]
++- SubqueryAlias spark_catalog.default.char_varchar_std_view_v
+ +- View (`spark_catalog`.`default`.`char_varchar_std_view_v`, [v#x])
+ +- Project [cast(v#x as varchar(5)) AS v#x]
+ +- Project [v#x]
+ +- SubqueryAlias spark_catalog.default.char_varchar_std
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
+
+
+-- !query
+WITH t AS (SELECT c, v FROM char_varchar_std) SELECT typeof(c), typeof(v) FROM t
+-- !query analysis
+WithCTE
+:- CTERelationDef xxxx, false
+: +- SubqueryAlias t
+: +- Project [c#x, v#x]
+: +- SubqueryAlias spark_catalog.default.char_varchar_std
+: +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+: +- Relation spark_catalog.default.char_varchar_std[c#x,v#x] parquet
++- Project [typeof(c#x) AS typeof(c)#x, typeof(v#x) AS typeof(v)#x]
+ +- SubqueryAlias t
+ +- CTERelationRef xxxx, true, [c#x, v#x], false, false
+
+
+-- !query
+CREATE TABLE char_varchar_std_orc (c CHAR(5), v VARCHAR(5)) USING orc
+-- !query analysis
+CreateDataSourceTableCommand `spark_catalog`.`default`.`char_varchar_std_orc`, false
+
+
+-- !query
+INSERT INTO char_varchar_std_orc VALUES ('ab', 'cd')
+-- !query analysis
+InsertIntoHadoopFsRelationCommand file:[not included in comparison]/{warehouse_dir}/char_varchar_std_orc, false, ORC, [path=file:[not included in comparison]/{warehouse_dir}/char_varchar_std_orc], Append, `spark_catalog`.`default`.`char_varchar_std_orc`, org.apache.spark.sql.execution.datasources.InMemoryFileIndex(file:[not included in comparison]/{warehouse_dir}/char_varchar_std_orc), [c, v]
++- Project [static_invoke(CharVarcharCodegenUtils.charTypeWriteSideCheck(cast(col1#x as char(5)), 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeWriteSideCheck(cast(col2#x as varchar(5)), 5)) AS v#x]
+ +- LocalRelation [col1#x, col2#x]
+
+
+-- !query
+SELECT typeof(c), typeof(v) FROM char_varchar_std_orc
+-- !query analysis
+Project [typeof(c#x) AS typeof(c)#x, typeof(v#x) AS typeof(v)#x]
++- SubqueryAlias spark_catalog.default.char_varchar_std_orc
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std_orc[c#x,v#x] orc
+
+
+-- !query
+SELECT concat('[', c, ']'), concat('[', v, ']') FROM char_varchar_std_orc
+-- !query analysis
+Project [concat([, cast(c#x as string), ]) AS concat([, c, ])#x, concat([, cast(v#x as string), ]) AS concat([, v, ])#x]
++- SubqueryAlias spark_catalog.default.char_varchar_std_orc
+ +- Project [static_invoke(CharVarcharCodegenUtils.charTypeReadSideCheck(c#x, 5)) AS c#x, static_invoke(CharVarcharCodegenUtils.varcharTypeReadSideCheck(v#x, 5)) AS v#x]
+ +- Relation spark_catalog.default.char_varchar_std_orc[c#x,v#x] orc
+
+
+-- !query
+DROP VIEW char_varchar_std_view_v
+-- !query analysis
+DropTableCommand `spark_catalog`.`default`.`char_varchar_std_view_v`, false, true, false
+
+
+-- !query
+DROP VIEW char_varchar_std_view
+-- !query analysis
+DropTableCommand `spark_catalog`.`default`.`char_varchar_std_view`, false, true, false
+
+
+-- !query
+DROP TABLE char_varchar_std_ctas
+-- !query analysis
+DropTable false, false
++- ResolvedIdentifier V2SessionCatalog(spark_catalog), default.char_varchar_std_ctas
+
+
+-- !query
+DROP TABLE char_varchar_std_orc
+-- !query analysis
+DropTable false, false
++- ResolvedIdentifier V2SessionCatalog(spark_catalog), default.char_varchar_std_orc
+
+
-- !query
DROP TABLE char_varchar_std
-- !query analysis
diff --git a/sql/core/src/test/resources/sql-tests/inputs/charvarchar-standard-semantics.sql b/sql/core/src/test/resources/sql-tests/inputs/charvarchar-standard-semantics.sql
index 25a52737bcb2..c48965f667fb 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/charvarchar-standard-semantics.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/charvarchar-standard-semantics.sql
@@ -182,6 +182,30 @@ SELECT cast('a' AS CHAR(2)) IN (cast('a' AS CHAR(4)), cast('b' AS VARCHAR(3)));
SELECT cast('a' AS CHAR(2) COLLATE UTF8_LCASE) = cast('a' AS VARCHAR(2) COLLATE UTF8_LCASE);
SELECT cast('a' AS CHAR(2) COLLATE UTF8_LCASE) IN (cast('a' AS VARCHAR(2) COLLATE UTF8_LCASE));
+-- CHAR/VARCHAR vs non-string follow STRING: compare promotes the string side to the other
+-- atomic type; COALESCE uses the same STRING promotion (including vs BOOLEAN).
+SELECT typeof(coalesce(cast('123' AS CHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS VARCHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS STRING), 1));
+SELECT coalesce(cast('123' AS CHAR(3)), 1),
+ coalesce(cast('123' AS VARCHAR(3)), 1),
+ coalesce(cast('123' AS STRING), 1);
+SELECT cast('123' AS CHAR(3)) = 123,
+ cast('123' AS VARCHAR(3)) = 123,
+ cast('123' AS STRING) = 123;
+SELECT typeof(coalesce(cast('1.5' AS CHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS VARCHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS STRING), 1.5));
+SELECT cast('2020-01-02' AS CHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS VARCHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS STRING) = date'2020-01-02';
+SELECT cast('true' AS CHAR(4)) = true,
+ cast('true' AS VARCHAR(4)) = true,
+ cast('true' AS STRING) = true;
+SELECT typeof(coalesce(cast('true' AS CHAR(4)), true));
+SELECT typeof(coalesce(cast('true' AS VARCHAR(4)), true));
+SELECT typeof(coalesce(cast('true' AS STRING), true));
+
-- Nested types keep CHAR/VARCHAR
SELECT typeof(array(cast('a' AS CHAR(2)), cast('bb' AS CHAR(3))));
SELECT typeof(struct(cast('a' AS CHAR(2)) AS f));
@@ -193,4 +217,22 @@ INSERT INTO char_varchar_std VALUES ('ab', 'ab');
SELECT typeof(c), typeof(v) FROM char_varchar_std;
SELECT concat('[', c, ']'), concat('[', v, ']') FROM char_varchar_std;
SELECT length(c), length(v) FROM char_varchar_std;
+
+-- Language surfaces: CTAS / VIEW inherit CHAR/VARCHAR; ORC catalog round-trip
+CREATE TABLE char_varchar_std_ctas USING parquet AS SELECT c, v FROM char_varchar_std;
+SELECT typeof(c), typeof(v) FROM char_varchar_std_ctas;
+CREATE VIEW char_varchar_std_view AS SELECT c FROM char_varchar_std;
+SELECT typeof(c) FROM char_varchar_std_view;
+CREATE VIEW char_varchar_std_view_v AS SELECT v FROM char_varchar_std;
+SELECT typeof(v) FROM char_varchar_std_view_v;
+WITH t AS (SELECT c, v FROM char_varchar_std) SELECT typeof(c), typeof(v) FROM t;
+CREATE TABLE char_varchar_std_orc (c CHAR(5), v VARCHAR(5)) USING orc;
+INSERT INTO char_varchar_std_orc VALUES ('ab', 'cd');
+SELECT typeof(c), typeof(v) FROM char_varchar_std_orc;
+SELECT concat('[', c, ']'), concat('[', v, ']') FROM char_varchar_std_orc;
+
+DROP VIEW char_varchar_std_view_v;
+DROP VIEW char_varchar_std_view;
+DROP TABLE char_varchar_std_ctas;
+DROP TABLE char_varchar_std_orc;
DROP TABLE char_varchar_std;
diff --git a/sql/core/src/test/resources/sql-tests/results/charvarchar-standard-semantics.sql.out b/sql/core/src/test/resources/sql-tests/results/charvarchar-standard-semantics.sql.out
index 168b5ef173be..538da5cffe49 100644
--- a/sql/core/src/test/resources/sql-tests/results/charvarchar-standard-semantics.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/charvarchar-standard-semantics.sql.out
@@ -750,6 +750,90 @@ struct<(CAST(a AS CHAR(2) COLLATE UTF8_LCASE) IN (CAST(a AS VARCHAR(2) COLLATE U
false
+-- !query
+SELECT typeof(coalesce(cast('123' AS CHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS VARCHAR(3)), 1)),
+ typeof(coalesce(cast('123' AS STRING), 1))
+-- !query schema
+struct<typeof(coalesce(CAST(123 AS CHAR(3)), 1)):string,typeof(coalesce(CAST(123 AS VARCHAR(3)), 1)):string,typeof(coalesce(CAST(123 AS STRING), 1)):string>
+-- !query output
+bigint bigint bigint
+
+
+-- !query
+SELECT coalesce(cast('123' AS CHAR(3)), 1),
+ coalesce(cast('123' AS VARCHAR(3)), 1),
+ coalesce(cast('123' AS STRING), 1)
+-- !query schema
+struct<coalesce(CAST(123 AS CHAR(3)), 1):bigint,coalesce(CAST(123 AS VARCHAR(3)), 1):bigint,coalesce(CAST(123 AS STRING), 1):bigint>
+-- !query output
+123 123 123
+
+
+-- !query
+SELECT cast('123' AS CHAR(3)) = 123,
+ cast('123' AS VARCHAR(3)) = 123,
+ cast('123' AS STRING) = 123
+-- !query schema
+struct<(CAST(123 AS CHAR(3)) = 123):boolean,(CAST(123 AS VARCHAR(3)) = 123):boolean,(CAST(123 AS STRING) = 123):boolean>
+-- !query output
+true true true
+
+
+-- !query
+SELECT typeof(coalesce(cast('1.5' AS CHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS VARCHAR(3)), 1.5)),
+ typeof(coalesce(cast('1.5' AS STRING), 1.5))
+-- !query schema
+struct<typeof(coalesce(CAST(1.5 AS CHAR(3)), 1.5)):string,typeof(coalesce(CAST(1.5 AS VARCHAR(3)), 1.5)):string,typeof(coalesce(CAST(1.5 AS STRING), 1.5)):string>
+-- !query output
+double double double
+
+
+-- !query
+SELECT cast('2020-01-02' AS CHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS VARCHAR(10)) = date'2020-01-02',
+ cast('2020-01-02' AS STRING) = date'2020-01-02'
+-- !query schema
+struct<(CAST(2020-01-02 AS CHAR(10)) = DATE '2020-01-02'):boolean,(CAST(2020-01-02 AS VARCHAR(10)) = DATE '2020-01-02'):boolean,(CAST(2020-01-02 AS STRING) = DATE '2020-01-02'):boolean>
+-- !query output
+true true true
+
+
+-- !query
+SELECT cast('true' AS CHAR(4)) = true,
+ cast('true' AS VARCHAR(4)) = true,
+ cast('true' AS STRING) = true
+-- !query schema
+struct<(CAST(true AS CHAR(4)) = true):boolean,(CAST(true AS VARCHAR(4)) = true):boolean,(CAST(true AS STRING) = true):boolean>
+-- !query output
+true true true
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS CHAR(4)), true))
+-- !query schema
+struct<typeof(coalesce(CAST(true AS CHAR(4)), true)):string>
+-- !query output
+boolean
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS VARCHAR(4)), true))
+-- !query schema
+struct<typeof(coalesce(CAST(true AS VARCHAR(4)), true)):string>
+-- !query output
+boolean
+
+
+-- !query
+SELECT typeof(coalesce(cast('true' AS STRING), true))
+-- !query schema
+struct<typeof(coalesce(CAST(true AS STRING), true)):string>
+-- !query output
+boolean
+
+
-- !query
SELECT typeof(array(cast('a' AS CHAR(2)), cast('bb' AS CHAR(3))))
-- !query schema
@@ -814,6 +898,126 @@ struct<length(c):int,length(v):int>
5 2
+-- !query
+CREATE TABLE char_varchar_std_ctas USING parquet AS SELECT c, v FROM char_varchar_std
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+SELECT typeof(c), typeof(v) FROM char_varchar_std_ctas
+-- !query schema
+struct<typeof(c):string,typeof(v):string>
+-- !query output
+char(5) varchar(5)
+
+
+-- !query
+CREATE VIEW char_varchar_std_view AS SELECT c FROM char_varchar_std
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+SELECT typeof(c) FROM char_varchar_std_view
+-- !query schema
+struct<typeof(c):string>
+-- !query output
+char(5)
+
+
+-- !query
+CREATE VIEW char_varchar_std_view_v AS SELECT v FROM char_varchar_std
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+SELECT typeof(v) FROM char_varchar_std_view_v
+-- !query schema
+struct<typeof(v):string>
+-- !query output
+varchar(5)
+
+
+-- !query
+WITH t AS (SELECT c, v FROM char_varchar_std) SELECT typeof(c), typeof(v) FROM t
+-- !query schema
+struct<typeof(c):string,typeof(v):string>
+-- !query output
+char(5) varchar(5)
+
+
+-- !query
+CREATE TABLE char_varchar_std_orc (c CHAR(5), v VARCHAR(5)) USING orc
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+INSERT INTO char_varchar_std_orc VALUES ('ab', 'cd')
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+SELECT typeof(c), typeof(v) FROM char_varchar_std_orc
+-- !query schema
+struct<typeof(c):string,typeof(v):string>
+-- !query output
+char(5) varchar(5)
+
+
+-- !query
+SELECT concat('[', c, ']'), concat('[', v, ']') FROM char_varchar_std_orc
+-- !query schema
+struct<concat([, c, ]):string,concat([, v, ]):string>
+-- !query output
+[ab ] [cd]
+
+
+-- !query
+DROP VIEW char_varchar_std_view_v
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+DROP VIEW char_varchar_std_view
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+DROP TABLE char_varchar_std_ctas
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+DROP TABLE char_varchar_std_orc
+-- !query schema
+struct<>
+-- !query output
+
+
+
-- !query
DROP TABLE char_varchar_std
-- !query schema
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
index 0c8008b5ddbf..c375694d7576 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
@@ -19,7 +19,7 @@ package org.apache.spark.sql
import scala.util.Try
-import org.apache.spark.{SparkConf, SparkException, SparkRuntimeException}
+import org.apache.spark.{SparkConf, SparkException, SparkRuntimeException, SparkThrowable}
import org.apache.spark.sql.catalyst.analysis.FunctionRegistry
import org.apache.spark.sql.catalyst.expressions.{Attribute, EqualTo, GreaterThan, Literal, ScalarSubquery, StringRPad}
import org.apache.spark.sql.catalyst.expressions.Cast.toSQLId
@@ -1292,6 +1292,61 @@ class BasicCharVarcharTestSuite extends SharedSparkSession {
}
}
+ test("SPARK-58794: CHAR/VARCHAR vs non-string follow STRING for compare and COALESCE") {
+ // Comparisons promote the string side to the other atomic type; COALESCE uses the same
+ // STRING promotion (and the same BOOLEAN path). CHAR/VARCHAR extend StringType, so they must
+ // match the STRING analogue on schema, values, and analysis errors. Exact-length CHAR keeps
+ // padding from confounding comparisons with the unpadded STRING value.
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
+ def followString(cvSql: String, strSql: String): Unit = {
+ val strResult = Try {
+ val df = sql(strSql)
+ (df.schema.map(_.dataType), df.collect().toSeq)
+ }
+ val cvResult = Try {
+ val df = sql(cvSql)
+ (df.schema.map(_.dataType), df.collect().toSeq)
+ }
+ (strResult, cvResult) match {
+ case (scala.util.Success((st, sr)), scala.util.Success((ct, cr))) =>
+ assert(ct === st, s"schema:\n $cvSql -> $ct\n $strSql -> $st")
+ assert(cr === sr, s"rows:\n $cvSql -> $cr\n $strSql -> $sr")
+ case (scala.util.Failure(eStr: SparkThrowable),
+ scala.util.Failure(eCv: SparkThrowable)) =>
+ assert(eCv.getCondition === eStr.getCondition,
+ s"$cvSql vs $strSql: ${eCv.getCondition} != ${eStr.getCondition}")
+ case (s, c) =>
+ fail(s"$cvSql vs $strSql: STRING success=${s.isSuccess} CV success=${c.isSuccess}")
+ }
+ }
+
+ val combos = Seq(
+ ("cast('123' AS CHAR(3))", "cast('123' AS VARCHAR(3))", "cast('123' AS STRING)", "123"),
+ ("cast('1.5' AS CHAR(3))", "cast('1.5' AS VARCHAR(3))", "cast('1.5' AS STRING)", "1.5"),
+ ("cast('2020-01-02' AS CHAR(10))", "cast('2020-01-02' AS VARCHAR(10))",
+ "cast('2020-01-02' AS STRING)", "date'2020-01-02'"),
+ ("cast('2020-01-02 03:04:05' AS CHAR(19))", "cast('2020-01-02 03:04:05' AS VARCHAR(19))",
+ "cast('2020-01-02 03:04:05' AS STRING)", "timestamp'2020-01-02 03:04:05'"),
+ ("cast('true' AS CHAR(4))", "cast('true' AS VARCHAR(4))", "cast('true' AS STRING)", "true"),
+ // Padded CHAR vs the same padded STRING / VARCHAR bytes.
+ ("cast('123' AS CHAR(5))", "cast('123 ' AS VARCHAR(5))", "cast('123 ' AS STRING)", "123"))
+
+ val templates: Seq[(String, String) => String] = Seq(
+ (cv, other) => s"SELECT typeof(coalesce($cv, $other))",
+ (cv, other) => s"SELECT coalesce($cv, $other)",
+ (cv, other) => s"SELECT $cv = $other",
+ (cv, other) => s"SELECT $cv < $other",
+ (cv, other) => s"SELECT $cv IN ($other)")
+
+ for ((charExpr, varcharExpr, stringExpr, other) <- combos) {
+ for (mk <- templates) {
+ followString(mk(charExpr, other), mk(stringExpr, other))
+ followString(mk(varcharExpr, other), mk(stringExpr, other))
+ }
+ }
+ }
+ }
+
test("SPARK-58794: parameterized CHAR/VARCHAR lengths under standardSemantics") {
// Length positions accept parameter markers (`integerValue` -> `parameterMarker`). Under
// standardSemantics the bound type stays first-class: CAST keeps CHAR/VARCHAR, pads and
@@ -1341,6 +1396,399 @@ class BasicCharVarcharTestSuite extends SharedSparkSession {
}
}
+ test("SPARK-58794: language surfaces keep CHAR/VARCHAR under standardSemantics") {
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
+ // CTAS / CREATE VIEW inherit projected CHAR/VARCHAR (D13).
+ withTable("std_src", "std_ctas") {
+ sql("CREATE TABLE std_src (c CHAR(5), v VARCHAR(5)) USING parquet")
+ sql("INSERT INTO std_src VALUES ('ab', 'ab')")
+ sql("CREATE TABLE std_ctas USING parquet AS SELECT c, v FROM std_src")
+ assert(spark.table("std_ctas").schema.map(_.dataType) ===
+ Seq(CharType(5), VarcharType(5)))
+ checkAnswer(
+ sql("SELECT concat('<', c, '>'), concat('<', v, '>') FROM std_ctas"),
+ Row("<ab >", "<ab>"))
+ val cteDf = sql(
+ "WITH t AS (SELECT c, v FROM std_src) SELECT typeof(c), typeof(v) FROM t")
+ checkAnswer(cteDf, Row("char(5)", "varchar(5)"))
+ }
+ withTable("std_view_src") {
+ withView("std_cv_view", "std_cv_view_v") {
+ sql("CREATE TABLE std_view_src (c CHAR(4), v VARCHAR(4)) USING parquet")
+ sql("INSERT INTO std_view_src VALUES ('xy', 'ab')")
+ sql("CREATE VIEW std_cv_view AS SELECT c FROM std_view_src")
+ sql("CREATE VIEW std_cv_view_v AS SELECT v FROM std_view_src")
+ assert(spark.table("std_cv_view").schema.head.dataType === CharType(4))
+ assert(spark.table("std_cv_view_v").schema.head.dataType === VarcharType(4))
+ checkAnswer(
+ sql("SELECT concat('<', c, '>') FROM std_cv_view"), Row("<xy >"))
+ checkAnswer(sql("SELECT v FROM std_cv_view_v"), Row("ab"))
+ }
+ }
+
+ // ALTER COLUMN equal-length CHAR/VARCHAR remains supported with first-class types.
+ // VARCHAR widen / CHAR->VARCHAR are allowed by CheckAnalysis on V2 tables
+ // (see DSV2CharVarcharDDLTestSuite); V1 file-source ALTER only evolves collation
+ // (same StringConstraint), so length changes stay rejected there.
+ withTable("std_alter") {
+ sql("CREATE TABLE std_alter (c CHAR(4), v VARCHAR(4)) USING parquet")
+ sql("ALTER TABLE std_alter CHANGE COLUMN c TYPE CHAR(4)")
+ sql("ALTER TABLE std_alter CHANGE COLUMN v TYPE VARCHAR(4)")
+ assert(spark.table("std_alter").schema.map(_.dataType) ===
+ Seq(CharType(4), VarcharType(4)))
+ intercept[AnalysisException] {
+ sql("ALTER TABLE std_alter CHANGE COLUMN c TYPE CHAR(5)")
+ }
+ intercept[AnalysisException] {
+ sql("ALTER TABLE std_alter CHANGE COLUMN v TYPE VARCHAR(5)")
+ }
+ }
+
+ // Session variables: DECLARE / SET keep the type and apply CAST assignment.
+ sql("DECLARE OR REPLACE VARIABLE std_char_var CHAR(4)")
+ sql("DECLARE OR REPLACE VARIABLE std_varchar_var VARCHAR(4)")
+ try {
+ sql("SET VARIABLE std_char_var = 'ab'")
+ val charVarDf = sql("SELECT std_char_var AS c")
+ assert(charVarDf.schema.head.dataType === CharType(4))
+ checkAnswer(sql("SELECT concat('<', std_char_var, '>')"), Row("<ab >"))
+ // Oversize by trailing blanks only is trimmed to fit CHAR(n).
+ sql("SET VARIABLE std_char_var = 'abcd '")
+ checkAnswer(sql("SELECT concat('<', std_char_var, '>')"), Row("<abcd>"))
+ intercept[SparkRuntimeException] {
+ sql("SET VARIABLE std_char_var = 'abcde'").collect()
+ }
+
+ sql("SET VARIABLE std_varchar_var = 'ab'")
+ val varcharVarDf = sql("SELECT std_varchar_var AS v")
+ assert(varcharVarDf.schema.head.dataType === VarcharType(4))
+ checkAnswer(sql("SELECT concat('<', std_varchar_var, '>')"), Row("<ab>"))
+ // Oversize by trailing blanks only is trimmed to fit.
+ sql("SET VARIABLE std_varchar_var = 'abcd '")
+ checkAnswer(sql("SELECT std_varchar_var"), Row("abcd"))
+ intercept[SparkRuntimeException] {
+ sql("SET VARIABLE std_varchar_var = 'abcde'").collect()
+ }
+ } finally {
+ sql("DROP TEMPORARY VARIABLE IF EXISTS std_char_var")
+ sql("DROP TEMPORARY VARIABLE IF EXISTS std_varchar_var")
+ }
+
+ // SQL scripting local variables keep CHAR/VARCHAR inside a compound statement.
+ val localVarScript =
+ """
+ |BEGIN
+ | DECLARE c CHAR(4);
+ | DECLARE v VARCHAR(4);
+ | SET c = 'ab';
+ | SET v = 'cd';
+ | SELECT typeof(c), concat('<', c, '>'), typeof(v), concat('<', v, '>');
+ |END
+ |""".stripMargin
+ val localVarDf = sql(localVarScript)
+ assert(localVarDf.schema.map(_.dataType) ===
+ Seq(StringType, StringType, StringType, StringType))
+ checkAnswer(localVarDf, Row("char(4)", "<ab >", "varchar(4)", "<cd>"))
+ // Trailing-blank trim on local SET into CHAR/VARCHAR.
+ checkAnswer(
+ sql(
+ """
+ |BEGIN
+ | DECLARE c CHAR(4);
+ | DECLARE v VARCHAR(4);
+ | SET c = 'abcd ';
+ | SET v = 'abcd ';
+ | SELECT concat('<', c, '>'), v;
+ |END
+ |""".stripMargin),
+ Row("<abcd>", "abcd"))
+ intercept[SparkRuntimeException] {
+ sql(
+ """
+ |BEGIN
+ | DECLARE c CHAR(4);
+ | SET c = 'abcde';
+ |END
+ |""".stripMargin).collect()
+ }
+ intercept[SparkRuntimeException] {
+ sql(
+ """
+ |BEGIN
+ | DECLARE v VARCHAR(4);
+ | SET v = 'abcde';
+ |END
+ |""".stripMargin).collect()
+ }
+
+ // Cursor FETCH INTO CHAR/VARCHAR locals applies store assignment (pad / length).
+ withSQLConf(SQLConf.SQL_SCRIPTING_CURSOR_ENABLED.key -> "true") {
+ val cursorScript =
+ """
+ |BEGIN
+ | DECLARE fetched_c CHAR(4);
+ | DECLARE fetched_v VARCHAR(4);
+ | DECLARE cur CURSOR FOR
+ | SELECT cast('ab' AS CHAR(4)) AS c, cast('cd' AS VARCHAR(4)) AS v;
+ | OPEN cur;
+ | FETCH cur INTO fetched_c, fetched_v;
+ | SELECT typeof(fetched_c), concat('<', fetched_c, '>'),
+ | typeof(fetched_v), concat('<', fetched_v, '>');
+ | CLOSE cur;
+ |END
+ |""".stripMargin
+ checkAnswer(
+ sql(cursorScript),
+ Row("char(4)", "<ab >", "varchar(4)", "<cd>"))
+
+ // FETCH plain STRING into CHAR pads via assignment cast.
+ checkAnswer(
+ sql(
+ """
+ |BEGIN
+ | DECLARE fetched CHAR(4);
+ | DECLARE cur CURSOR FOR SELECT 'ab' AS c;
+ | OPEN cur;
+ | FETCH cur INTO fetched;
+ | SELECT typeof(fetched), concat('<', fetched, '>');
+ | CLOSE cur;
+ |END
+ |""".stripMargin),
+ Row("char(4)", "<ab >"))
+
+ // FETCH into a wider CHAR pads; trailing blanks trim into a shorter target.
+ checkAnswer(
+ sql(
+ """
+ |BEGIN
+ | DECLARE fetched CHAR(5);
+ | DECLARE cur CURSOR FOR SELECT cast('xy' AS CHAR(2)) AS c;
+ | OPEN cur;
+ | FETCH cur INTO fetched;
+ | SELECT concat('<', fetched, '>');
+ | CLOSE cur;
+ |END
+ |""".stripMargin),
+ Row("<xy >"))
+ checkAnswer(
+ sql(
+ """
+ |BEGIN
+ | DECLARE fetched_c CHAR(4);
+ | DECLARE fetched_v VARCHAR(4);
+ | DECLARE cur CURSOR FOR
+ | SELECT cast('abcd ' AS CHAR(5)) AS c, cast('abcd ' AS VARCHAR(5)) AS v;
+ | OPEN cur;
+ | FETCH cur INTO fetched_c, fetched_v;
+ | SELECT concat('<', fetched_c, '>'), fetched_v;
+ | CLOSE cur;
+ |END
+ |""".stripMargin),
+ Row("<abcd>", "abcd"))
+ intercept[SparkRuntimeException] {
+ sql(
+ """
+ |BEGIN
+ | DECLARE fetched VARCHAR(2);
+ | DECLARE cur CURSOR FOR SELECT cast('abcd' AS VARCHAR(4)) AS v;
+ | OPEN cur;
+ | FETCH cur INTO fetched;
+ | CLOSE cur;
+ |END
+ |""".stripMargin).collect()
+ }
+ }
+
+ // SQL FUNCTION params/RETURNS apply store assignment (pad, blank-trim, overflow).
+ sql("CREATE OR REPLACE TEMPORARY FUNCTION std_char_fn() RETURNS CHAR(3) RETURN 'a'")
+ sql(
+ """CREATE OR REPLACE TEMPORARY FUNCTION std_varchar_ret()
+ |RETURNS VARCHAR(3) RETURN 'ab'""".stripMargin)
+ sql(
+ """CREATE OR REPLACE TEMPORARY FUNCTION std_char_param(x CHAR(3))
+ |RETURNS CHAR(3) RETURN x""".stripMargin)
+ sql(
+ """CREATE OR REPLACE TEMPORARY FUNCTION std_varchar_param(x VARCHAR(3))
+ |RETURNS VARCHAR(3) RETURN x""".stripMargin)
+ try {
+ val fnDf = sql("SELECT std_char_fn() AS c")
+ assert(fnDf.schema.head.dataType === CharType(3))
+ checkAnswer(sql("SELECT concat('<', std_char_fn(), '>')"), Row("<a >"))
+ val retVarcharDf = sql("SELECT std_varchar_ret() AS v")
+ assert(retVarcharDf.schema.head.dataType === VarcharType(3))
+ checkAnswer(retVarcharDf, Row("ab"))
+
+ // STRING -> CHAR(n) param: pad; trailing blanks trim; non-blank overflow errors.
+ val charParamDf = sql("SELECT std_char_param('a') AS c")
+ assert(charParamDf.schema.head.dataType === CharType(3))
+ checkAnswer(sql("SELECT concat('<', std_char_param('a'), '>')"), Row("<a >"))
+ checkAnswer(
+ sql("SELECT concat('<', std_char_param('abc '), '>')"),
+ Row("<abc>"))
+ intercept[SparkRuntimeException] {
+ sql("SELECT std_char_param('abcd')").collect()
+ }
+
+ val paramDf = sql("SELECT std_varchar_param('ab') AS v")
+ assert(paramDf.schema.head.dataType === VarcharType(3))
+ checkAnswer(paramDf, Row("ab"))
+ checkAnswer(sql("SELECT std_varchar_param('abc ')"), Row("abc"))
+ intercept[SparkRuntimeException] {
+ sql("SELECT std_varchar_param('abcd')").collect()
+ }
+ } finally {
+ sql("DROP TEMPORARY FUNCTION IF EXISTS std_char_fn")
+ sql("DROP TEMPORARY FUNCTION IF EXISTS std_varchar_ret")
+ sql("DROP TEMPORARY FUNCTION IF EXISTS std_char_param")
+ sql("DROP TEMPORARY FUNCTION IF EXISTS std_varchar_param")
+ }
+
+ // ORC catalog tables stamp the catalyst type so typeof survives write/read.
+ withTable("std_orc") {
+ sql("CREATE TABLE std_orc (c CHAR(5), v VARCHAR(5)) USING orc")
+ sql("INSERT INTO std_orc VALUES ('ab', 'cd')")
+ assert(spark.table("std_orc").schema.map(_.dataType) ===
+ Seq(CharType(5), VarcharType(5)))
+ checkAnswer(
+ sql("SELECT concat('<', c, '>'), concat('<', v, '>') FROM std_orc"),
+ Row("<ab >", "<cd>"))
+ }
+
+ // File-only ORC inference recovers the catalyst type stamped on write.
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ spark.range(1).selectExpr("cast('ab' AS CHAR(4)) AS c")
+ .write.mode("overwrite").orc(path)
+ val orcDf = spark.read.orc(path)
+ assert(orcDf.schema.head.dataType === CharType(4))
+ checkAnswer(orcDf.selectExpr("concat('<', c, '>')"), Row("<ab >"))
+ // Reading with first-class types off replaces CHAR with STRING even if the
+ // file was stamped under standardSemantics.
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "false") {
+ val readOff = spark.read.orc(path)
+ assert(readOff.schema.head.dataType === StringType)
+ }
+ }
+ // First-class types off: CAST CHAR is STRING before the writer, so ORC does not stamp CHAR.
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "false") {
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ spark.range(1).selectExpr("cast('ab' AS CHAR(4)) AS c")
+ .write.mode("overwrite").orc(path)
+ assert(spark.read.orc(path).schema.head.dataType === StringType)
+ }
+ }
+ // preserveCharVarcharTypeInfo also keeps first-class types, so write still stamps CHAR.
+ withSQLConf(
+ SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "false",
+ SQLConf.PRESERVE_CHAR_VARCHAR_TYPE_INFO.key -> "true") {
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ spark.range(1).selectExpr("cast('ab' AS CHAR(4)) AS c")
+ .write.mode("overwrite").orc(path)
+ assert(spark.read.orc(path).schema.head.dataType === CharType(4))
+ }
+ }
+ // ORC stamps collated unbounded STRING as plain "string"; the inferred type is the
+ // same as Avro, which omits the catalyst property on unbounded STRING.
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ spark.range(1).selectExpr("cast('ab' AS STRING COLLATE UTF8_LCASE) AS c")
+ .write.mode("overwrite").orc(path)
+ assert(spark.read.orc(path).schema.head.dataType === StringType)
+ }
+
+ // Avro conversion stamps CHAR/VARCHAR (including nested fields and CHAR map keys).
+ // The avro data source lives in connector/avro; sql/core still owns toAvroType,
+ // serializer, and deserializer, so round-trip CHAR/VARCHAR here with a DataFileWriter.
+ val converters = org.apache.spark.sql.avro.SchemaConverters
+ Seq(CharType(5), VarcharType(7)).foreach { dt =>
+ val avro = converters.toAvroType(dt, nullable = false)
+ val back = converters.toSqlType(avro).dataType
+ assert(back === dt, s"Avro round-trip lost $dt, got $back")
+ }
+ val nestedSchema = new StructType()
+ .add("c", CharType(4))
+ .add("s", new StructType().add("f", VarcharType(3)))
+ .add("m", MapType(CharType(2), VarcharType(3)))
+ val nestedAvro = converters.toAvroType(nestedSchema, nullable = false)
+ assert(converters.toSqlType(nestedAvro).dataType === nestedSchema)
+ val badStamp = org.apache.avro.SchemaBuilder.builder().stringType()
+ badStamp.addProp("spark.sql.catalyst.type", "int")
+ val badStampErr = intercept[Exception] {
+ converters.toSqlType(badStamp)
+ }
+ assert(badStampErr.getMessage.contains("STRING subtype") ||
+ Option(badStampErr.getCause).exists(_.getMessage.contains("STRING subtype")))
+ // Flag-off replace happens before Avro sees the type, so CHAR is not stamped.
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "false") {
+ val replaced = CharVarcharUtils.replaceCharVarcharWithString(CharType(4))
+ assert(replaced === StringType)
+ val replacedAvro = converters.toAvroType(replaced, nullable = false)
+ assert(replacedAvro.getProp("spark.sql.catalyst.type") === null)
+ assert(converters.toSqlType(replacedAvro).dataType === StringType)
+ }
+
+ withTempPath { file =>
+ val ser = new org.apache.spark.sql.avro.AvroSerializer(
+ nestedSchema, nestedAvro, nullable = false)
+ val row = org.apache.spark.sql.catalyst.InternalRow(
+ org.apache.spark.unsafe.types.UTF8String.fromString("ab "),
+ org.apache.spark.sql.catalyst.InternalRow(
+ org.apache.spark.unsafe.types.UTF8String.fromString("xy")),
+ new org.apache.spark.sql.catalyst.util.ArrayBasedMapData(
+ new org.apache.spark.sql.catalyst.util.GenericArrayData(Array(
+ org.apache.spark.unsafe.types.UTF8String.fromString("k "))),
+ new org.apache.spark.sql.catalyst.util.GenericArrayData(Array(
+ org.apache.spark.unsafe.types.UTF8String.fromString("v")))))
+ val record = ser.serialize(row)
+ .asInstanceOf[org.apache.avro.generic.GenericRecord]
+ val writer = new org.apache.avro.file.DataFileWriter(
+ new org.apache.avro.generic.GenericDatumWriter[org.apache.avro.generic.GenericRecord](
+ nestedAvro))
+ writer.create(nestedAvro, file)
+ writer.append(record)
+ writer.close()
+ val reader = new org.apache.avro.file.DataFileReader(
+ file,
+ new org.apache.avro.generic.GenericDatumReader[org.apache.avro.generic.GenericRecord]())
+ try {
+ assert(converters.toSqlType(reader.getSchema).dataType === nestedSchema)
+ val deser = new org.apache.spark.sql.avro.AvroDeserializer(
+ nestedAvro, nestedSchema, "CORRECTED", false, "", -1)
+ val back = deser.deserialize(reader.next()).get
+ .asInstanceOf[org.apache.spark.sql.catalyst.InternalRow]
+ assert(back.getUTF8String(0).toString === "ab ")
+ val nestedField = back.getStruct(1, 1)
+ assert(nestedField.getUTF8String(0).toString === "xy")
+ val mapData = back.getMap(2)
+ assert(mapData.numElements() === 1)
+ assert(mapData.keyArray().getUTF8String(0).toString === "k ")
+ assert(mapData.valueArray().getUTF8String(0).toString === "v")
+ } finally {
+ reader.close()
+ }
+ }
+
+ // JSON / CSV keep a user-specified CHAR/VARCHAR schema under the flag.
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ spark.range(1).selectExpr("cast(id AS STRING) AS c").write.mode("overwrite")
+ .json(s"$path/json")
+ val jsonDf = spark.read.schema("c CHAR(5)").json(s"$path/json")
+ assert(jsonDf.schema.head.dataType === CharType(5))
+ checkAnswer(jsonDf.selectExpr("concat('<', c, '>')"), Row("<0 >"))
+
+ spark.range(1).selectExpr("cast(id AS STRING) AS c").write.mode("overwrite")
+ .option("header", "true").csv(s"$path/csv")
+ val csvDf = spark.read.schema("c VARCHAR(5)").option("header", "true")
+ .csv(s"$path/csv")
+ assert(csvDf.schema.head.dataType === VarcharType(5))
+ checkAnswer(csvDf, Row("0"))
+ }
+ }
+ }
+
test("SPARK-58802: single-pass resolver agrees with fixed-point under standardSemantics") {
// Dual run defaults to on under tests, but pin it explicitly so this coverage cannot be
// silently lost: the HybridAnalyzer compares output schema and normalized plan across the
@@ -1709,6 +2157,55 @@ class FileSourceCharVarcharTestSuite extends CharVarcharTestSuite with SharedSpa
}
}
+ test("SPARK-58794: EXTERNAL TABLE CHAR/VARCHAR pad, assignment, and oversize") {
+ // D14: external file tables use the same store assignment on write and pad +
+ // length-check on scan. Hive TRANSFORM is out of scope for this epic.
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ // Pre-existing short file values: CHAR pads on scan; types stay first-class.
+ sql("SELECT 'ab' AS c, 'cd' AS v").write.format(format).save(path)
+ withTable("std_ext") {
+ sql(
+ s"""CREATE EXTERNAL TABLE std_ext (c CHAR(5), v VARCHAR(5))
+ |USING $format LOCATION '$path'""".stripMargin)
+ assert(spark.table("std_ext").schema.map(_.dataType) ===
+ Seq(CharType(5), VarcharType(5)))
+ checkAnswer(
+ sql("SELECT concat('<', c, '>'), concat('<', v, '>') FROM std_ext"),
+ Row("<ab >", "<cd>"))
+
+ // INSERT into EXTERNAL applies write-side assignment (pad / length).
+ sql("INSERT INTO std_ext VALUES ('x', 'yz')")
+ checkAnswer(
+ sql("SELECT concat('<', c, '>'), concat('<', v, '>') FROM std_ext " +
+ "WHERE c LIKE 'x%'"),
+ Row("<x >", "<yz>"))
+ intercept[SparkRuntimeException] {
+ sql("INSERT INTO std_ext VALUES ('too-long', 'ok')").collect()
+ }
+ }
+ }
+ // Bypass-writer oversize non-blank values fail on scan (D12).
+ withTempPath { dir =>
+ val path = dir.getCanonicalPath
+ sql("SELECT 'abcdef' AS c").write.format(format).save(path)
+ withTable("std_ext_oversize") {
+ sql(
+ s"""CREATE EXTERNAL TABLE std_ext_oversize (c CHAR(3))
+ |USING $format LOCATION '$path'""".stripMargin)
+ checkError(
+ exception = intercept[SparkRuntimeException] {
+ sql("SELECT * FROM std_ext_oversize").collect()
+ },
+ condition = "EXCEED_LIMIT_LENGTH",
+ parameters = Map("limit" -> "3")
+ )
+ }
+ }
+ }
+ }
+
test("alter table set location w/ fit length values") {
withTempPath { dir =>
withTable("t") {
@@ -1840,6 +2337,27 @@ class DSV2CharVarcharTestSuite extends CharVarcharTestSuite
.set(SQLConf.DEFAULT_CATALOG.key, "testcat")
}
+ test("SPARK-58794: VARCHAR widen and CHAR->VARCHAR under standardSemantics") {
+ // V2 CheckAnalysis allows length-preserving CHAR->VARCHAR and VARCHAR widen;
+ // V1 file-source ALTER does not (collation-only evolution).
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
+ withTable("std_v2_alter") {
+ sql(s"CREATE TABLE std_v2_alter (c CHAR(4), v VARCHAR(4)) USING $format")
+ sql("ALTER TABLE std_v2_alter CHANGE COLUMN v TYPE VARCHAR(5)")
+ assert(spark.table("std_v2_alter").schema("v").dataType === VarcharType(5))
+ sql("ALTER TABLE std_v2_alter CHANGE COLUMN c TYPE VARCHAR(5)")
+ assert(spark.table("std_v2_alter").schema.map(_.dataType) ===
+ Seq(VarcharType(5), VarcharType(5)))
+ intercept[AnalysisException] {
+ sql("ALTER TABLE std_v2_alter CHANGE COLUMN v TYPE VARCHAR(4)")
+ }
+ intercept[AnalysisException] {
+ sql("ALTER TABLE std_v2_alter CHANGE COLUMN c TYPE CHAR(5)")
+ }
+ }
+ }
+ }
+
test("char/varchar type values length check: partitioned columns of other types") {
Seq("CHAR(5)", "VARCHAR(5)").foreach { typ =>
withTable("t") {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org