[GitHub] [druid] clintropolis commented on pull request #11018: add protobuf inputformat

[GitBox <gi...@apache.org>]

clintropolis commented on pull request #11018:
URL: https://github.com/apache/druid/pull/11018#issuecomment-803271847


   >@clintropolis Hi, I create ProtobufInputFormat followed your suggestion. I don't use this interface before, so I'm not very familiar with it. Can you review my code and tell me if this implementation meet requirement or not? If this implementation is correct, I will add more unit tests. By the way, where should I change in document about this feature?
   
   Thanks! I will have a look this weekend. I think https://github.com/apache/druid/blob/master/docs/ingestion/data-formats.md is the appropriate place to document the new `InputFormat` (I guess we also forgot to update the protobuf section of this in the last PR, https://github.com/apache/druid/blob/master/docs/ingestion/data-formats.md#protobuf-parser)
   
   >@clintropolis I reviewed code about Avro for inputFormat and learnt it only supports batch ingestion jobs. Why do we not support stream ingestion jobs? I think it's not very hard to implement it and I'm glad to do that.
   
   :+1: The only reason streaming Avro isn't supported yet is basically the same reason it wasn't done for Protobuf, simply that no one has done the conversion. I think it would be great if you would like to take that on, especially since I think Avro and Protobuf (until this PR) are the only "core" extensions that do not yet support `InputFormat`. It would make ingestion be consistent for native batch and streaming, and be much appreciated!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org