[jira] [Created] (IGNITE-4140) KafkaStreamer should use tuple extractor instead of decoders

["Valentin Kulichenko (JIRA)" <ji...@apache.org>]

Valentin Kulichenko created IGNITE-4140:
-------------------------------------------

             Summary: KafkaStreamer should use tuple extractor instead of decoders
                 Key: IGNITE-4140
                 URL: https://issues.apache.org/jira/browse/IGNITE-4140
             Project: Ignite
          Issue Type: Improvement
          Components: streaming
    Affects Versions: 1.7
            Reporter: Valentin Kulichenko
             Fix For: 2.0


Current design of {{KafkaStreamer}} looks incorrect to me. In particular, it extends {{StreamAdapter}}, but ignores tuple extractors provided there and uses native Kafka decoders instead. This for example makes impossible to produce several entries from one message, like it can be done via {{StreamMultipleTupleExtractor}} in other streamers.

To fix this, we should:
# Declare the {{KafkaStreamer}} like this:
{code}
KafkaStreamer<K, V> extends StreamAdapter<MessageAndMetadata<byte[], byte[]>, K, V>
{code}
# Remove {{keyDecoder}} and {{valDecoder}} in favor of tuple extractors.
# Instead of doing {{getStreamer().addData(...)}} directly, call {{addMessage(...)}} method providing the raw message consumed from Kafka ({{MessageAndMetadata<byte[], byte[]>}}). This method will make sure that configured extractor is invoked and that all entries are added to {{IgniteDataStreamer}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)