[jira] [Created] (AVRO-2904) timestamp-millis doesn't truncate when in a union with null

["Dan Lipofsky (Jira)" <ji...@apache.org>]

Dan Lipofsky created AVRO-2904:
----------------------------------

             Summary: timestamp-millis doesn't truncate when in a union with null
                 Key: AVRO-2904
                 URL: https://issues.apache.org/jira/browse/AVRO-2904
             Project: Apache Avro
          Issue Type: Bug
          Components: java, logical types
    Affects Versions: 1.10.0
            Reporter: Dan Lipofsky


In Avro 1.10.0 the setter for logical-type timestamp-millis will truncate the timestamp to millis in the simplest condition but not when it is in a union with null.

It looks like something similar was addressed in AVRO-2360 but that was supposedly fixed in 1.9.0 and this is still broken in 1.10.0.

Here is my IDL
{noformat}
  record TimestampTest1 {
    union { null, timestamp_ms } ts;
  }
  record TimestampTest2 {
    timestamp_ms ts;
  }
{noformat}
Here is my test, in which {{TimestampTest2}} passes but {{TimestampTest1}} fails.
{noformat}
public class SerDeTest {
    @Test
    public void TimestampTest1() throws IOException {
        final TimestampTest1 ts = TimestampTest1.newBuilder().setTs(Instant.now()).build();
        byte[] bytes = serialize(ts);
        TimestampTest1 other = deserialize(ts.getSchema(), bytes);
        Assert.assertEquals(ts, other);
    }
    @Test
    public void TimestampTest2() throws IOException {
        final TimestampTest2 ts = TimestampTest2.newBuilder().setTs(Instant.now()).build();
        byte[] bytes = serialize(ts);
        TimestampTest2 other = deserialize(ts.getSchema(), bytes);
        Assert.assertEquals(ts, other);
    }

    private <T extends SpecificRecord> T deserialize(Schema schema, byte[] bytes)
            throws IOException {
        Decoder decoder = DecoderFactory.get().binaryDecoder(bytes, null);
        SpecificDatumReader<T> reader = new SpecificDatumReader<>(schema);
        return reader.read(null, decoder);
    }

    private byte[] serialize(SpecificRecord record) throws IOException {
        try (final ByteArrayOutputStream out = new ByteArrayOutputStream()) {
            SpecificDatumWriter<SpecificRecord> writer = new SpecificDatumWriter<>(
                record.getSchema());
            Encoder encoder = EncoderFactory.get().binaryEncoder(out, null);
            writer.write(record, encoder);
            encoder.flush();
            return out.toByteArray();
        }
    }
}
{noformat}
{{TimestampTest1}} fails with
{noformat}
java.lang.AssertionError: 
Expected :{"ts": 2020-07-23T21:35:12.348379Z}
Actual   :{"ts": 2020-07-23T21:35:12.348Z}
{noformat}
If I change my tests to use {{Instant.now().truncatedTo(ChronoUnit.MILLIS)}} they'll both pass, but it seems like that should not be necessary, particularly given the code below.

Looking at the generated Java DTO, I see a few differences, but the most important would seem to be that the working DTO has
{noformat}
  public void setTs(java.time.Instant value) {
    this.ts = value.truncatedTo(java.time.temporal.ChronoUnit.MILLIS);
  }
{noformat}
while the broken one has
{noformat}
  public void setTs(java.time.Instant value) {
    this.ts = value;
  }
{noformat}
The working one also has this code, which is missing from the broken one:
{noformat}
  private static final org.apache.avro.Conversion<?>[] conversions =
      new org.apache.avro.Conversion<?>[] {
      new org.apache.avro.data.TimeConversions.TimestampMillisConversion(),
      null
  };

  @Override
  public org.apache.avro.Conversion<?> getConversion(int field) {
    return conversions[field];
  }
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)