You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Takuya Ueshin (Jira)" <ji...@apache.org> on 2024/10/21 23:03:00 UTC

[jira] [Created] (SPARK-50063) Support Variant in Spark Connect Scala client

Takuya Ueshin created SPARK-50063:
-------------------------------------

             Summary: Support Variant in Spark Connect Scala client
                 Key: SPARK-50063
                 URL: https://issues.apache.org/jira/browse/SPARK-50063
             Project: Spark
          Issue Type: Sub-task
          Components: Connect
    Affects Versions: 4.0.0
            Reporter: Takuya Ueshin


Currently Spark Connect Scala client doesn't support Variant type when collecting data.

{code:scala}
scala> spark.sql("select parse_json('1')").collect()
org.apache.spark.SparkUnsupportedOperationException: [UNSUPPORTED_DATATYPE] Unsupported data type "VARIANT". SQLSTATE: 0A000
  org.apache.spark.sql.errors.ExecutionErrors.unsupportedDataTypeError(ExecutionErrors.scala:159)
  org.apache.spark.sql.errors.ExecutionErrors.unsupportedDataTypeError$(ExecutionErrors.scala:156)
  org.apache.spark.sql.errors.ExecutionErrors$.unsupportedDataTypeError(ExecutionErrors.scala:254)
  org.apache.spark.sql.connect.client.arrow.ArrowDeserializers$.deserializerFor(ArrowDeserializer.scala:370)
...
 {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org