[jira] [Commented] (AVRO-2799) Endless recursive calls by using ProtobufDatumReader when decoding proto map

["Tianyu Lang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082682#comment-17082682 ] 

Tianyu Lang commented on AVRO-2799:
-----------------------------------

I discovered the similar issue: https://issues.apache.org/jira/browse/AVRO-2780

Do you have "Outer class" in your Protobuf definition? Basically an outer class is created when you have a .proto file named "A", and inside you also have a message named "A", then in the generated code, you will have an "outer class" for A.

> Endless recursive calls by using ProtobufDatumReader when decoding proto map
> ----------------------------------------------------------------------------
>
>                 Key: AVRO-2799
>                 URL: https://issues.apache.org/jira/browse/AVRO-2799
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.9.2
>            Reporter: Allen Wang
>            Priority: Major
>
> I have a protobuf message defined as this:
>  
> {code:java}
> syntax = "proto3";
> package ExampleProtobuf;
> option java_package = "com.example";
> import "google/protobuf/any.proto";
> import "google/protobuf/timestamp.proto";
> import "google/protobuf/wrappers.proto";
> enum EventType {
>   TYPE_UNKNOWN = 0;
>   TYPE_ORDER_RECEIVED = 1;
>   TYPE_PAYMENT_AUTHORIZED = 2;
>   TYPE_PAYMENT_CAPTURED = 3;
>   TYPE_ORDER_SUBMITTED = 4;
>   TYPE_ORDER_DELIVERY_CREATED = 5;
>   TYPE_MERCHANT_CONFIRM_ORDER = 6;
> }
> message RevenueEvent {
>   google.protobuf.StringValue id = 1;
>   EventType type = 2;
>   google.protobuf.Timestamp change_time = 3;
>   google.protobuf.StringValue country = 4;
>   map <string, google.protobuf.Any> payload = 5;
>   google.protobuf.StringValue source = 6;
>   bool is_test = 7;
>   google.protobuf.StringValue version = 8;
>   google.protobuf.StringValue checksum = 9;
> }
> {code}
>  
>  
> I wrote a program to encode such Protobuf message with Avro schema and convert it back to Protobuf message:
>  
>  
> {code:java}
> public class EventTest {
>   public static void main(String[] args) throws  Exception {
>     Example.RevenueEvent event = RevenueEvent.newBuilder()
>             .setId(StringValue.newBuilder()
>                     .setValue("12345")
>                     .build())
>             .setChangeTime(Timestamp.newBuilder()
>                     .setSeconds(System.currentTimeMillis())
>                     .build())
>             .setChecksum(StringValue.newBuilder()
>                     .setValue("xyz")
>                     .build())
>             .setCountry(StringValue.newBuilder()
>                     .setValue("US")
>                     .build())
>             .setSource(StringValue.newBuilder()
>                     .setValue("unknown")
>                     .build())
>             .setType(EventType.TYPE_MERCHANT_CONFIRM_ORDER)
>             .putPayload("key", Any.newBuilder().setValue(ByteString.copyFromUtf8("xyz")).build())
>             .build();
>     ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
>     Encoder encoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
>     GenericDatumWriter<Example.RevenueEvent> datumWriter = new ProtobufDatumWriter<Example.RevenueEvent>(RevenueEvent.class);
>     datumWriter.write(event, encoder);
>     encoder.flush();
>     byte[] bytes = outputStream.toByteArray();
>     DatumReader<RevenueEvent> datumReader = new ProtobufDatumReader<>(RevenueEvent.class);
>     InputStream inputStream = new ByteArrayInputStream(bytes);
>     Decoder decoder = DecoderFactory.get().binaryDecoder(inputStream, null);
>     datumReader.read(null, decoder);
>   }
> }
> {code}
>  
>  
> The program crashed with java.lang.StackOverflowError:
>  
> Exception in thread "main" java.lang.StackOverflowErrorException in thread "main" java.lang.StackOverflowError at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) at org.apache.avro.specific.SpecificData.getClass(SpecificData.java:250) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:141) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143)
> ...
>  
> Looking into ProtoData.java, there seems to be a situation where the recursion would not end:
>  
> {code:java}
> @Override
> public Object newRecord(Object old, Schema schema) {
>   try {
>     Class c = SpecificData.get().getClass(schema);
>     if (c == null)
>       return newRecord(old, schema); // punt to generic <-- endless recursion
>     if (c.isInstance(old))
>       return old; // reuse instance
>     return c.getMethod("newBuilder").invoke(null);
>   } catch (Exception e) {
>     throw new RuntimeException(e);
>   }
> }
> {code}
> Further experiment shows that if the map payload is not set, the decoding would be successful. 
>  
> protobuf-java:3.8.0 is used.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)