[jira] [Assigned] (SPARK-58218) [SQL] Replace generic RuntimeException with typed exceptions in AvroDeserializer

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-58218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon reassigned SPARK-58218:
------------------------------------

    Assignee: Ganesha S

> [SQL] Replace generic RuntimeException with typed exceptions in AvroDeserializer
> --------------------------------------------------------------------------------
>
>                 Key: SPARK-58218
>                 URL: https://issues.apache.org/jira/browse/SPARK-58218
>             Project: Spark
>          Issue Type: Task
>          Components: SQL
>    Affects Versions: 4.2.0
>            Reporter: Ganesha S
>            Assignee: Ganesha S
>            Priority: Major
>              Labels: pull-request-available
>
> `AvroDeserializer` throws three generic `java.lang.RuntimeExceptions` where the rest of the class, and the rest of Spark, use typed exceptions. This makes the failures uncatchable by error class, gives them no SQLSTATE, and, for one of the sites, is inconsistent with the sibling cases in the same method.
> The three sites on the current master (sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala):
> - Line 278 — a value that is not a valid Avro binary during BYTES -> BinaryType conversion:
> {code:scala}
> case other =>
>   throw new RuntimeException(errorPrefix + s"$other is not a valid avro binary.")
> {code}
> Every other type-mismatch case in the same newWriter method (lines 187, 202, 212, 224, 233) throws IncompatibleSchemaException(errorPrefix + ...). Only this one uses a bare RuntimeException.
> - Line 332 — a null element read into a non-nullable array element type:
> {code:scala}
> throw new RuntimeException(
>   s"Array value at path ${toFieldStr(avroElementPath)} is not allowed to be null")
> {code}
> - Line 364 — a null value read into a non-nullable map value type:
> {code:scala}
> throw new RuntimeException(
>   s"Map value at path ${toFieldStr(avroPath :+ "value")} is not allowed to be null")
> {code}
> *Why this is a problem*
> Spark has largely migrated runtime failures to the error-class framework (SparkRuntimeException, SparkException.internalError, named error conditions with SQLSTATE). Generic RuntimeExceptions at these sites:
> - cannot be caught or matched on an error condition by callers,
> - carry no SQLSTATE, and
> - (line 278) are inconsistent with the five sibling cases in the same method that already use IncompatibleSchemaException.
> Notably, the write path already has a typed, catalogued error for the analogous situation: AVRO_CANNOT_WRITE_NULL_FIELD (SQLSTATE 22004). The read path has no equivalent.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org