[jira] [Commented] (AVRO-2788) Generated Avro schema from Protobuf is missing default values for repeated fields

["Doug Cutting (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073176#comment-17073176 ] 

Doug Cutting commented on AVRO-2788:
------------------------------------

A Github pull request is preferred to a patch file.  Repo is [https://github.com/apache/avro].

This patch requires manual regeneration of the protobuf test sources.  Thus the submitted changes should also include these generated files, since they're managed under source control.

To get tests to pass I also had to edit lang/java/pom.xml, changing the spotless excludes from "Test.java" to "**".

Thanks!

 

> Generated Avro schema from Protobuf is missing default values for repeated fields
> ---------------------------------------------------------------------------------
>
>                 Key: AVRO-2788
>                 URL: https://issues.apache.org/jira/browse/AVRO-2788
>             Project: Apache Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.9.2
>            Reporter: Tianyu Lang
>            Priority: Major
>         Attachments: AVRO-2788.patch
>
>
> Avro schemas generated from Protobuf schemas by *ProtobufData.java* are missing default values for repeated (array) fields.
> This will break compatibility when Avro is used as a transport format between 2 services that use Protobuf internally.
>  
> For example:
> A publisher converts Protobuf to Avro, then sends the message through Kafka to a consumer. The consumer then converts Avro back into Protobuf, then does all the processing with Protobuf.
>  
> A compatibility issue will occur when a new repeated Protobuf field is added to the consumer Protobuf schema. The corresponding Avro schema generated from
> {code:java}
> Schema schema = ProtobufData.get().getSchema(MyProtobufClass.class);
> {code}
>  will not assign default values to the newly added repeated field. Because the publisher is still on the schema without the newly added array field, deserialization on the consumer side will fail since there is no default values to fill in.
>  
>  
>  
> I discussed this with [~cutting] on the mailing list and it makes sense to just add default values for Protobuf repeated fields.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)