[jira] [Commented] (HUDI-73) Support vanilla Avro Kafka Source in HoodieDeltaStreamer

["Pratyaksh Sharma (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092682#comment-17092682 ] 

Pratyaksh Sharma commented on HUDI-73:
--------------------------------------

Solving this has the following pros and cons - 

Pros -> 
 * schema-registry set up is not needed for deserializing avro data. For deserializing, user can simply use source schema file. 

Cons -> 
 * User will not be able to leverage the benefit of schema evolution using avro and schema-registry. While ingesting, one would have to be sure that there are no schema changes coming in the incoming batch and that even if there are any, the same can be handled with the source schema file provided.

> Support vanilla Avro Kafka Source in HoodieDeltaStreamer
> --------------------------------------------------------
>
>                 Key: HUDI-73
>                 URL: https://issues.apache.org/jira/browse/HUDI-73
>             Project: Apache Hudi (incubating)
>          Issue Type: New Feature
>          Components: DeltaStreamer
>            Reporter: Balaji Varadarajan
>            Priority: Major
>
> Context : [https://github.com/uber/hudi/issues/597]
> Currently, Avro Kafka Source expects the installation to use Confluent version with SchemaRegistry server running. We need to support the Kafka installations which do not use Schema Registry by allowing FileBasedSchemaProvider to be integrated to AvroKafkaSource.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)