[jira] [Created] (AVRO-2799) Endless recursive calls by using ProtobufDatumReader when decoding proto map

["Allen Wang (Jira)" <ji...@apache.org>]

Allen Wang created AVRO-2799:
--------------------------------

             Summary: Endless recursive calls by using ProtobufDatumReader when decoding proto map
                 Key: AVRO-2799
                 URL: https://issues.apache.org/jira/browse/AVRO-2799
             Project: Apache Avro
          Issue Type: Bug
          Components: java
    Affects Versions: 1.9.2
            Reporter: Allen Wang


I have a protobuf message defined as this:

 
{code:java}
syntax = "proto3";

package ExampleProtobuf;

option java_package = "com.example";

import "google/protobuf/any.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";

enum EventType {
  TYPE_UNKNOWN = 0;
  TYPE_ORDER_RECEIVED = 1;
  TYPE_PAYMENT_AUTHORIZED = 2;
  TYPE_PAYMENT_CAPTURED = 3;
  TYPE_ORDER_SUBMITTED = 4;
  TYPE_ORDER_DELIVERY_CREATED = 5;
  TYPE_MERCHANT_CONFIRM_ORDER = 6;
}

message RevenueEvent {
  google.protobuf.StringValue id = 1;
  EventType type = 2;
  google.protobuf.Timestamp change_time = 3;
  google.protobuf.StringValue country = 4;
  map <string, google.protobuf.Any> payload = 5;
  google.protobuf.StringValue source = 6;
  bool is_test = 7;
  google.protobuf.StringValue version = 8;
  google.protobuf.StringValue checksum = 9;
}
{code}
 

 

I wrote a program to encode such Protobuf message with Avro schema and convert it back to Protobuf message:

 

 
{code:java}
public class EventTest {
  public static void main(String[] args) throws  Exception {
    Example.RevenueEvent event = RevenueEvent.newBuilder()
            .setId(StringValue.newBuilder()
                    .setValue("12345")
                    .build())
            .setChangeTime(Timestamp.newBuilder()
                    .setSeconds(System.currentTimeMillis())
                    .build())
            .setChecksum(StringValue.newBuilder()
                    .setValue("xyz")
                    .build())
            .setCountry(StringValue.newBuilder()
                    .setValue("US")
                    .build())
            .setSource(StringValue.newBuilder()
                    .setValue("unknown")
                    .build())
            .setType(EventType.TYPE_MERCHANT_CONFIRM_ORDER)
            .putPayload("key", Any.newBuilder().setValue(ByteString.copyFromUtf8("xyz")).build())
            .build();
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    Encoder encoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
    GenericDatumWriter<Example.RevenueEvent> datumWriter = new ProtobufDatumWriter<Example.RevenueEvent>(RevenueEvent.class);
    datumWriter.write(event, encoder);
    encoder.flush();
    byte[] bytes = outputStream.toByteArray();

    DatumReader<RevenueEvent> datumReader = new ProtobufDatumReader<>(RevenueEvent.class);
    InputStream inputStream = new ByteArrayInputStream(bytes);
    Decoder decoder = DecoderFactory.get().binaryDecoder(inputStream, null);
    datumReader.read(null, decoder);
  }
}
{code}
 

 

The program crashed with java.lang.StackOverflowError:

 

Exception in thread "main" java.lang.StackOverflowErrorException in thread "main" java.lang.StackOverflowError at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) at org.apache.avro.specific.SpecificData.getClass(SpecificData.java:250) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:141) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143)

...

 

Looking into ProtoData.java, there seems to be a situation where the recursion would not end:

 
{code:java}
@Override
public Object newRecord(Object old, Schema schema) {
  try {
    Class c = SpecificData.get().getClass(schema);
    if (c == null)
      return newRecord(old, schema); // punt to generic <-- endless recursion
    if (c.isInstance(old))
      return old; // reuse instance
    return c.getMethod("newBuilder").invoke(null);

  } catch (Exception e) {
    throw new RuntimeException(e);
  }
}

{code}
Further experiment shows that if the map payload is not set, the decoding would be successful. 

 

protobuf-java:3.8.0 is used.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)