You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "cloud-fan (via GitHub)" <gi...@apache.org> on 2026/07/13 16:30:32 UTC
[PR] [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228 [spark]
cloud-fan opened a new pull request, #57228:
URL: https://github.com/apache/spark/pull/57228
### What changes were proposed in this pull request?
Replace the legacy error condition `_LEGACY_ERROR_TEMP_2228` with a descriptive condition `UDT_CLASS_NOT_FOUND` (SQLSTATE `38000`), part of the error-class migration (umbrella [SPARK-37935](https://issues.apache.org/jira/browse/SPARK-37935)).
- Add `UDT_CLASS_NOT_FOUND` to `error-conditions.json` with two subclasses and remove `_LEGACY_ERROR_TEMP_2228`:
- `FOR_USER_CLASS` — the UDT class registered for a user class via `UDTRegistration` cannot be loaded (replaces `_LEGACY_ERROR_TEMP_2228`).
- `WITHOUT_USER_CLASS` — the UDT class named directly in a schema string cannot be loaded.
- `DataTypeErrors.cannotLoadUserDefinedTypeError` now raises `UDT_CLASS_NOT_FOUND.FOR_USER_CLASS`.
- `DataType.fromJson`'s Scala/Java UDT branch now catches `ClassNotFoundException` from `SparkClassUtils.classForName` and rewraps it as `UDT_CLASS_NOT_FOUND.WITHOUT_USER_CLASS`. Previously a raw `java.lang.ClassNotFoundException` (no error class, no SQLSTATE) propagated when a schema string (for example the schema stored in Parquet/ORC file metadata, or a Delta table's stored schema) referenced a UDT class that was not on the classpath.
- Add tests in `DataTypeSuite` (`WITHOUT_USER_CLASS`) and `UDTRegistrationSuite` (`FOR_USER_CLASS`).
### Why are the changes needed?
Part of the error-class migration ([SPARK-37935](https://issues.apache.org/jira/browse/SPARK-37935)). The legacy condition used free-form `name`/`userClass` parameters and, for the `DataType.fromJson` path, a missing UDT class was not wrapped at all: a raw `ClassNotFoundException` propagated without any error class or SQLSTATE, so it could not be identified or handled programmatically. Assigning a proper name and SQLSTATE, and wrapping the `fromJson` path, makes both failures diagnosable.
### Does this PR introduce _any_ user-facing change?
Yes, within the unreleased `master` branch only. The error condition name and SQLSTATE change (`_LEGACY_ERROR_TEMP_2228` -> `UDT_CLASS_NOT_FOUND.FOR_USER_CLASS`), and loading a non-existent UDT class named in a schema string now raises `UDT_CLASS_NOT_FOUND.WITHOUT_USER_CLASS` (SQLSTATE `38000`) instead of a raw `ClassNotFoundException`.
### How was this patch tested?
New unit tests:
- `DataTypeSuite`: "UDT_CLASS_NOT_FOUND.WITHOUT_USER_CLASS when UDT class is not on classpath".
- `UDTRegistrationSuite`: "UDT_CLASS_NOT_FOUND.FOR_USER_CLASS when UDT class is not loadable".
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228 [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #57228:
URL: https://github.com/apache/spark/pull/57228#issuecomment-4975899342
thanks for the review, merging to master/4.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228 [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #57228:
URL: https://github.com/apache/spark/pull/57228#discussion_r3576792183
##########
sql/api/src/main/scala/org/apache/spark/sql/types/DataType.scala:
##########
@@ -345,7 +345,15 @@ object DataType {
}
// Defense in depth: resolve the class without initializing it and verify that it really is a
// UserDefinedType subclass before constructing it.
- val clazz = SparkClassUtils.classForName[UserDefinedType[_]](udtClass, initialize = false)
+ val clazz = try {
+ SparkClassUtils.classForName[UserDefinedType[_]](udtClass, initialize = false)
+ } catch {
+ case e: ClassNotFoundException =>
+ throw new SparkClassNotFoundException(
+ errorClass = "UDT_CLASS_NOT_FOUND.WITHOUT_USER_CLASS",
+ messageParameters = Map("udtClass" -> udtClass),
+ cause = e)
+ }
Review Comment:
The Scala linter CI check (`Linters, licenses, and dependencies`) is failing on scalafmt here — `Scalafmt: Unformatted files found` in the `spark-sql-api` module. scalafmt wants the `try` on its own line under the assignment (and the body re-indented). Applying:
```suggestion
val clazz =
try {
SparkClassUtils.classForName[UserDefinedType[_]](udtClass, initialize = false)
} catch {
case e: ClassNotFoundException =>
throw new SparkClassNotFoundException(
errorClass = "UDT_CLASS_NOT_FOUND.WITHOUT_USER_CLASS",
messageParameters = Map("udtClass" -> udtClass),
cause = e)
}
```
(Or just run `./dev/scalafmt`.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228 [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #57228:
URL: https://github.com/apache/spark/pull/57228#issuecomment-4976050858
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/014aaf8648ef45a027bcfd554268fd9a64f7c10c
- merged into branch-4.x https://github.com/apache/spark/commit/abb6a8731561699a8a8fcd532919303d231a6f32
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228 [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #57228: [SPARK-58103][SQL] Assign a name to the error condition _LEGACY_ERROR_TEMP_2228
URL: https://github.com/apache/spark/pull/57228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org