[GitHub] [beam] ccciudatu edited a comment on pull request #13428: [BEAM-11338] Beam schema for thrift data

[GitBox <gi...@apache.org>]

ccciudatu edited a comment on pull request #13428:
URL: https://github.com/apache/beam/pull/13428#issuecomment-742657119


   @TheNeuralBit I'm trying to use KafkaIO to handle thrift input and output with some beam "canonical" processing on Rows in between.
   In my opinion, the first step towards simplifying the use of the various schema providers available would be to allow registering the top of the class hierarchy with a given provider, rather than every individual subtype used with a pipeline. E.g.
   
       pipeline.getSchemaRegistry().registerSchemaProvider(Message.class, new ProtoMessageSchema());
       pipeline.getSchemaRegistry().registerSchemaProvider(TBase.class, ThriftSchema.provider());
   
   Looking up the schema for any proto message type should resolve to what's registered for `Message`, unless something more specific is defined for a given type.
   I'd be glad to give it a try, if you think this is worth it.
   
   P.S.
   I added some test cases for thrift "default values" in this PR, along with the proper union mapping that I mentioned. Sorry for the plethora of amends and force pushes, I just wanted to keep all changes since your last review in one commit.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org