You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/04/03 14:31:41 UTC
[jira] [Commented] (FLINK-3871) Add Kafka TableSource with Avro
serialization
[ https://issues.apache.org/jira/browse/FLINK-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953573#comment-15953573 ]
ASF GitHub Bot commented on FLINK-3871:
---------------------------------------
GitHub user twalthr opened a pull request:
https://github.com/apache/flink/pull/3663
[FLINK-3871] [table] Add Kafka TableSource with Avro serialization
This PR adds KafkaAvroTableSource. It serializes/deserializes (nested) Avro records to (nested) Flink rows. Avro Utf8 strings are converted to regular Java strings.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/flink FLINK-3871
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3663.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3663
----
commit 589e45c5c50c328783f71d219c6606e972f42f34
Author: twalthr <tw...@apache.org>
Date: 2017-04-03T12:44:46Z
[FLINK-3871] [table] Add Kafka TableSource with Avro serialization
----
> Add Kafka TableSource with Avro serialization
> ---------------------------------------------
>
> Key: FLINK-3871
> URL: https://issues.apache.org/jira/browse/FLINK-3871
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
> Reporter: Fabian Hueske
> Assignee: Ivan Mushketyk
>
> Add a Kafka TableSource which supports Avro serialized data.
> The KafkaAvroTableSource should support two modes:
> # SpecificRecord Mode: In this case the user specifies a class which was code-generated by Avro depending on a schema. Flink treats these classes as regular POJOs. Hence, they are also natively supported by the Table API and SQL. Classes generated by Avro contain their Schema in a static field. The schema should be used to automatically derive field names and types. Hence, there is no additional information required than the name of the class.
> # GenericRecord Mode: In this case the user specifies an Avro Schema. The schema is used to deserialize the data into a GenericRecord which must be translated into possibly nested {{Row}} based on the schema information. Again, the Avro Schema is used to automatically derive the field names and types. This mode is less efficient than the SpecificRecord mode because the {{GenericRecord}} needs to be converted into {{Row}}.
> This feature depends on FLINK-5280, i.e., support for nested data in {{TableSource}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)