You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by dw...@apache.org on 2018/08/08 07:26:29 UTC
[flink] branch release-1.6 updated: [FLINK-9438] Add documentation
for AvroDeserializationSchema (#6078)
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a commit to branch release-1.6
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.6 by this push:
new 2dc6e0b [FLINK-9438] Add documentation for AvroDeserializationSchema (#6078)
2dc6e0b is described below
commit 2dc6e0be93d6a0c593d59a3b0fc887fa39a3b2b9
Author: Dawid Wysakowicz <wy...@gmail.com>
AuthorDate: Wed Aug 8 09:24:46 2018 +0200
[FLINK-9438] Add documentation for AvroDeserializationSchema (#6078)
---
docs/dev/connectors/kafka.md | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/docs/dev/connectors/kafka.md b/docs/dev/connectors/kafka.md
index 47a6651..b9dbd8f 100644
--- a/docs/dev/connectors/kafka.md
+++ b/docs/dev/connectors/kafka.md
@@ -166,6 +166,39 @@ For convenience, Flink provides the following schemas:
The KeyValue objectNode contains a "key" and "value" field which contain all fields, as well as
an optional "metadata" field that exposes the offset/partition/topic for this message.
+3. `AvroDeserializationSchema` which reads data serialized with Avro format using a statically provided schema. It can
+ infer the schema from Avro generated classes (`AvroDeserializationSchema.forSpecific(...)`) or it can work with `GenericRecords`
+ with a manually provided schema (with `AvroDeserializationSchema.forGeneric(...)`). This deserialization schema expects that
+ the serialized records DO NOT contain embedded schema.
+
+ - There is also a version of this schema available that can lookup the writer's schema (schema which was used to write the record) in
+ [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/docs/index.html). Using these deserialization schema
+ record will be read with the schema that was retrieved from Schema Registry and transformed to a statically provided( either through
+ `ConfluentRegistryAvroDeserializationSchema.forGeneric(...)` or `ConfluentRegistryAvroDeserializationSchema.forSpecific(...)`).
+
+ <br>To use this deserialization schema one has to add the following additional dependency:
+
+<div class="codetabs" markdown="1">
+<div data-lang="AvroDeserializationSchema" markdown="1">
+{% highlight xml %}
+<dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-avro</artifactId>
+ <version>{{site.version }}</version>
+</dependency>
+{% endhighlight %}
+</div>
+<div data-lang="ConfluentRegistryAvroDeserializationSchema" markdown="1">
+{% highlight xml %}
+<dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-avro-confluent-registry</artifactId>
+ <version>{{site.version }}</version>
+</dependency>
+{% endhighlight %}
+</div>
+</div>
+
When encountering a corrupted message that cannot be deserialized for any reason, there
are two options - either throwing an exception from the `deserialize(...)` method
which will cause the job to fail and be restarted, or returning `null` to allow