You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/08/31 15:08:45 UTC

[jira] [Commented] (FLINK-2372) Update KafkaSink to use new Producer API

    [ https://issues.apache.org/jira/browse/FLINK-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723382#comment-14723382 ] 

ASF GitHub Bot commented on FLINK-2372:
---------------------------------------

GitHub user rmetzger opened a pull request:

    https://github.com/apache/flink/pull/1082

    [FLINK-2372] Add new FlinkKafkaProducer

    This pull request is reworking Flink's KafkaSink.
    
    I've renamed the class to `FlinkKafkaProducer` to match it with `FlinkKafkaConsumer`.
    It uses the new Kafka [Producer API](http://kafka.apache.org/documentation.html#producerapi).
    In the documentation, they recommend using the new Producer API:
    
    > As of the 0.8.2 release we encourage all new development to use the new Java producer. This client is production tested and generally both faster and more fully featured than the previous Scala client.
    
    I've also noticed a good performance gain while doing some benchmarks with Flink and Kafka.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rmetzger/flink flink2372-second

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1082.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1082
    
----
commit f042a6471f300e09894c521678a60aa144790134
Author: Robert Metzger <rm...@apache.org>
Date:   2015-08-28T12:33:49Z

    [FLINK-2372] Add new FlinkKafkaProducer

----


> Update KafkaSink to use new Producer API
> ----------------------------------------
>
>                 Key: FLINK-2372
>                 URL: https://issues.apache.org/jira/browse/FLINK-2372
>             Project: Flink
>          Issue Type: Task
>          Components: Kafka Connector
>    Affects Versions: 0.10
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>             Fix For: 0.10
>
>
> Flink's Kafka Sink is using Kafka's old Producer API, which is has very poor performance and a limited API.
> I'll implement a new KafkaSink which is using the new API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)