You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/01/06 14:46:40 UTC
[jira] [Commented] (FLINK-3058) Add Kafka consumer for new 0.9.0.0
Kafka API
[ https://issues.apache.org/jira/browse/FLINK-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085546#comment-15085546 ]
ASF GitHub Bot commented on FLINK-3058:
---------------------------------------
GitHub user rmetzger opened a pull request:
https://github.com/apache/flink/pull/1489
[FLINK-3058] Add support for Kafka 0.9.0.0
For adding Kafka 0.9.0.0 support, this commit changes the following:
- Split up of the kafka connector into a flink-connector-kafka-(base|0.9|0.8) with different dependencies
- The base package contains common test cases and implementations (for example the producer for 0.9 and 0.8 relies on exactly the same code)
- the 0.8 package contains a kafka connector implementation against the SimpleConsumer (low level) API of Kafka 0.8. There are some additional tests for the ZK offset committing
- The 0.9 package relies on the new Consumer API of Kafka 0.9.0.0
- Support for metrics for all producers and the 0.9 consumer through Flink's accumulators.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/flink flink3058-second-rebased-rebased
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1489.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1489
----
commit d1a1659029b246cc164fe3cb274b02d87696e679
Author: Robert Metzger <rm...@apache.org>
Date: 2015-12-16T16:29:42Z
[FLINK-3058] Add support for Kafka 0.9.0.0
For adding Kafka 0.9.0.0 support, this commit changes the following:
- Split up of the kafka connector into a flink-connector-kafka-(base|0.9|0.8) with different dependencies
- The base package contains common test cases, classes and implementations (the producer for 0.9 and 0.8 relies on exactly the same code)
- the 0.8 package contains a kafka connector implementation against the SimpleConsumer (low level) API of Kafka 0.8. There are some additional tests for the ZK offset committing
- The 0.9 package relies on the new Consumer API of Kafka 0.9.0.0
- Support for metrics for all producers and the 0.9 consumer through Flink's accumulators.
----
> Add Kafka consumer for new 0.9.0.0 Kafka API
> --------------------------------------------
>
> Key: FLINK-3058
> URL: https://issues.apache.org/jira/browse/FLINK-3058
> Project: Flink
> Issue Type: New Feature
> Components: Kafka Connector
> Affects Versions: 1.0.0
> Reporter: Robert Metzger
> Assignee: Robert Metzger
>
> The Apache Kafka project is about to release a new consumer API . They also changed their internal protocol so Kafka 0.9.0.0 users will need an updated consumer from Flink.
> Also, I would like to let Flink be among the first stream processors supporting Kafka 0.9.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)