You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Anish Shrigondekar (Jira)" <ji...@apache.org> on 2026/02/23 18:25:00 UTC
[jira] [Resolved] (SPARK-55062) from_protobuf and to_protobuf do not support proto2 extensions
[ https://issues.apache.org/jira/browse/SPARK-55062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anish Shrigondekar resolved SPARK-55062.
----------------------------------------
Fix Version/s: 4.2.0
Resolution: Fixed
Issue resolved by pull request 53828
[https://github.com/apache/spark/pull/53828]
> from_protobuf and to_protobuf do not support proto2 extensions
> --------------------------------------------------------------
>
> Key: SPARK-55062
> URL: https://issues.apache.org/jira/browse/SPARK-55062
> Project: Spark
> Issue Type: Improvement
> Components: Protobuf
> Affects Versions: 4.1.1
> Reporter: David Young
> Assignee: David Young
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.2.0
>
>
> Proto2 extensions are currently silently dropped by from_protobuf and to_protobuf.
> This can be reproduced with any toy proto file containing an extension. For example, given a message `Foo` with regular field `a` and extension field `b`, from_protobuf will read only `a`.
> This is because the dynamic message used for serde is created without an extension registry. The fix will entail iterating through all file descriptors in the descriptor set and finding all extensions applicable to the target message. We cannot support extensions when we are not provided a file descriptor set, as the Java class for the message will only have the descriptor for the file it is defined in.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org