You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "jiayuasu (via GitHub)" <gi...@apache.org> on 2026/02/10 08:09:23 UTC
[PR] [GH-2608] Fix RasterUDT JSON schema serialization for Delta/Parquet write [sedona]
jiayuasu opened a new pull request, #2636:
URL: https://github.com/apache/sedona/pull/2636
## What changes were proposed in this pull request?
Add a `jsonValue` override to `RasterUDT` that strips the trailing `$` from the Scala case object class name, fixing Delta Lake and Parquet write failures.
## Why are the changes needed?
`RasterUDT` is defined as a Scala `case object`, so `getClass.getName` returns `org.apache.spark.sql.sedona_sql.UDT.RasterUDT$` (with a trailing `$`). Spark's `UserDefinedType.jsonValue` stores this class name in the JSON schema. When Delta Lake or Parquet tries to reconstruct the UDT during deserialization via `Class.forName(...).getConstructor().newInstance()`, it fails with:
- `NoSuchMethodException: RasterUDT$.<init>()` (JSON schema round-trip)
- `UNSUPPORTED_DATATYPE` referencing `RasterUDT$` (Parquet/Delta write)
This is the same issue that was previously fixed in `GeometryUDT\This is the raphyUDT`.
Note: `RS_Union_Aggr` was not affected because it uses `ExpressionEncoder` resolved via `UDTRegistration`, which stores `classOf[RasterUDT].getName` (without the `$` suffix). Other raster functions (e.g., `RS_MakeEmptyRaster`) use `InferredExpression` which references the `case object` singleton directly.
## How was this patch tested?
Added 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAdded 3 new tests to `RasAddede written to and read from Parquet (this already worked before the fix, serving as a control test)
All tests pass after the fix. Tests 1 and 2 fail withoutAll tests pass after the fix. Tests 1 and 2 fail withoutAll tests pass after the fix. Tsedona-1.9.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2608] Fix RasterUDT JSON schema serialization for Delta/Parquet write [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu merged PR #2636:
URL: https://github.com/apache/sedona/pull/2636
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2608] Fix RasterUDT JSON schema serialization for Delta/Parquet write [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu closed pull request #2636: [GH-2608] Fix RasterUDT JSON schema serialization for Delta/Parquet write
URL: https://github.com/apache/sedona/pull/2636
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org