You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "vinoyang (Jira)" <ji...@apache.org> on 2021/08/30 02:03:00 UTC
[jira] [Closed] (HUDI-2320) Add support ByteArrayDeserializer in
AvroKafkaSource
[ https://issues.apache.org/jira/browse/HUDI-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
vinoyang closed HUDI-2320.
--------------------------
Resolution: Done
bf5a52e51bbeaa089995335a0a4c55884792e505
> Add support ByteArrayDeserializer in AvroKafkaSource
> ----------------------------------------------------
>
> Key: HUDI-2320
> URL: https://issues.apache.org/jira/browse/HUDI-2320
> Project: Apache Hudi
> Issue Type: Improvement
> Components: DeltaStreamer
> Reporter: 董可伦
> Assignee: 董可伦
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> When the 'value.serializer' of Kafka Avro Producer is 'org.apache.kafka.common.serialization.ByteArraySerializer',Use the following configuration
> {code:java}
> --source-class org.apache.hudi.utilities.sources.AvroKafkaSource \
> --schemaprovider-class org.apache.hudi.utilities.schema.JdbcbasedSchemaProvider \
> --hoodie-conf "hoodie.deltastreamer.source.kafka.value.deserializer.class=org.apache.kafka.common.serialization.ByteArrayDeserializer"
> {code}
> For now,It will throw an exception::
> {code:java}
> java.lang.ClassCastException: [B cannot be cast to org.apache.avro.generic.GenericRecord{code}
> After support ByteArrayDeserializer,Use the configuration above，It works properly.And there is no need to provide 'schema.registry.url',For example, we can use the JdbcbasedSchemaProvider to get the sourceSchema
--
This message was sent by Atlassian Jira
(v8.3.4#803005)