You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/03/18 22:24:00 UTC
[jira] [Updated] (SPARK-56043) Wrap NullPointerException from Avro 1.12.x ParseContext.resolve() in SchemaParseException
[ https://issues.apache.org/jira/browse/SPARK-56043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-56043:
-----------------------------------
Labels: pull-request-available (was: )
> Wrap NullPointerException from Avro 1.12.x ParseContext.resolve() in SchemaParseException
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-56043
> URL: https://issues.apache.org/jira/browse/SPARK-56043
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 4.1.0, 4.0.0, 4.0.1, 4.0.2, 4.2.0, 4.1.1, 4.1.2, 4.0.3
> Reporter: Boyang Jerry Peng
> Priority: Major
> Labels: pull-request-available
>
> After the Avro 1.11.3 → 1.12.0 upgrade (SPARK-49014), {{Schema.Parser().parse()}} can throw {{NullPointerException}} from {{ParseContext.resolve()}} for certain invalid user-provided schemas. In Avro 1.11.x, the same schemas threw {{{}SchemaParseException{}}}, which was caught by the existing {{NonFatal(e)}} handler in {{AvroDataToCatalyst.nullSafeEval}} and reported as {{{}MALFORMED_AVRO_MESSAGE{}}}. The NPE bypasses nothing — it's still {{NonFatal}} — but it surfaces as an unclassified internal error rather than a schema parse error because it was never expected from schema parsing.
>
> *Root cause:* AVRO-3666 replaced the {{Names}} lookup (which threw {{{}SchemaParseException("Undefined name: ..."){}}}) with {{{}ParseContext.resolve(){}}}, which calls {{Objects.requireNonNull(oldSchemas.get(fullName))}} — throwing a raw {{NullPointerException}} when named types can't be resolved.
> *Verified behavioral difference* (standalone test against both jars):
> ||Schema pattern||Avro 1.11.3||Avro 1.12.1||
> |Undefined named type in record field|{{SchemaParseException}}|{{AvroTypeException}}|
> |Undefined type in union|{{SchemaParseException}}|{{AvroTypeException}}|
> |Bare string reference ({{{}"com.test.Missing"{}}})|{{SchemaParseException}}|*{{NullPointerException}}*|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org