[GitHub] [beam] mosche commented on pull request #24992: [Avro] Use "extensions/avro" instead of avro from"core" in Java SDK modules

["mosche (via GitHub)" <gi...@apache.org>]

mosche commented on PR #24992:
URL: https://github.com/apache/beam/pull/24992#issuecomment-1411492609

   > We can keep it "as it is" for now but it will be anyway a breaking change once Avro will be dropped from "core". Do you see any other options?
   
   @aromanenko-dev You don't need Avro to produce Avro compatible bytes. It's a well defined format with a detailed Spec. Of course this could become a hassle quickly, but luckily this case is trivial. Avro uses [varint zigzag encoding for longs](https://avro.apache.org/docs/1.8.2/spec.html#binary_encoding), that's just the same Protobuf is using for [signed ints](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-ints). And Avro records are just a concatenation of their fields without any further additions:
   
   Here's an example using Protobufs `CodedOutputStream`
   ```Java
   //import org.apache.beam.vendor.grpc.v1p48p1.com.google.protobuf.CodedOutputStream
   CodedOutputStream cos = CodedOutputStream.newInstance(outputStream);
   cos.writeSInt64NoTag(mark.getLastEmitted()); // signed int64 with varint zigzag encoding
   cos.writeSInt64NoTag(mark.getStartTime().getMillis()); // signed int64 with varint zigzag encoding
   cos.flush();
   ```
   
   Of course, that would require some additional tests to verify compatibility...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org