You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@ignite.apache.org by "Roman Shtykh (JIRA)" <ji...@apache.org> on 2016/03/18 08:06:33 UTC

[jira] [Comment Edited] (IGNITE-2730) Ignite Events Source Streaming to Kafka

    [ https://issues.apache.org/jira/browse/IGNITE-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201089#comment-15201089 ] 

Roman Shtykh edited comment on IGNITE-2730 at 3/18/16 7:05 AM:
---------------------------------------------------------------

Some implementation details for reviewers:
- I use a distributed queue as a buffer that is polled by Kafka. This is done to keep cache event data safe between polls.
- Schema facilities of Kafka Connect are not used -- cache events are marshalled with _JDKMarshaller_ and sent to Kafka. Later they can be deserialized with the provided _CacheEventDeserializer_, as done in the test code.
- Kafka partition keys are not used in the current implementation. If there are requests from users, we can enhance.


was (Author: roman_s):
Some implementation details for reviewers:
- I use a distributed queue as a buffer that is polled by Kafka. This is done to keep cache event data safe between polls.
- Schema facilities of Kafka Connect are not used -- cache events are marshalled with _JDKMarshaller_ and sent to Kafka. Later they can be deserialized with the provided _CacheEventDeserializer_, as done in the test code.
- Kafka partition keys are specified in the current implementation. If there are requests from users, we can enhance.

> Ignite Events Source Streaming to Kafka
> ---------------------------------------
>
>                 Key: IGNITE-2730
>                 URL: https://issues.apache.org/jira/browse/IGNITE-2730
>             Project: Ignite
>          Issue Type: New Feature
>          Components: streaming
>            Reporter: Roman Shtykh
>            Assignee: Roman Shtykh
>              Labels: community
>
> Streaming specified Ignite events (https://apacheignite.readme.io/docs/events) to Kafka via Kafka Connect.
> It has to be added to org.apache.ignite.stream.kafka.connect package.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)