You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Nikolai Tikhonov <nt...@apache.org> on 2017/06/05 11:58:25 UTC
Re: Best way to send records to Kafka from DataStreamer Receiver
Hi,
You can avoid sending message to local lister. From datastreamer receiver
you can submit a closure (which will send a message to kafka) to another
thread pool.
On Wed, May 31, 2017 at 4:13 PM, fatality <fa...@gmail.com> wrote:
> Hi
>
> I am using IgniteDataStreamer to take records and process them via the
> receiver in IgniteDataStreamer and as a result of the process, for some
> records, I would like to send messages into another Kafka Topic inside the
> receiver. Could you please advice a good way to do that or maybe this is
> already possible by using some common library in ignite?
>
> Currently i am using ignite messaging to send records into the
> locallisteners of the datastreamer.But i am not sure if that is a good idea
> once i have huge traffic.
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Best-way-to-send-records-to-Kafka-from-
> DataStreamer-Receiver-tp13291.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Best way to send records to Kafka from DataStreamer Receiver
Posted by fatih tekin <fa...@gmail.com>.
Thank you, Nikolay, for your support and kindness.
On Mon, Jun 5, 2017 at 5:33 PM, Nikolai Tikhonov <nt...@apache.org>
wrote:
> I'm glad to hear that you were able to solve this problem yourself! If you
> will face with some problems feel free to ask.
>
> Thanks,
> Nikolay
>
> On Mon, Jun 5, 2017 at 3:28 PM, fatih tekin <fa...@gmail.com>
> wrote:
>
>> Hi Nikolai,
>> Thanks for responding, That is good idea but we are having some metric
>> for the kafka messages and i don't want to put the our common jar into the
>> classpath of the ignite. For now, I am sending all the messages in one go
>> to as Json String. So that also reduces the network round as well.
>>
>> On Mon, Jun 5, 2017 at 1:58 PM, Nikolai Tikhonov <nt...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> You can avoid sending message to local lister. From datastreamer
>>> receiver you can submit a closure (which will send a message to kafka) to
>>> another thread pool.
>>>
>>> On Wed, May 31, 2017 at 4:13 PM, fatality <fa...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> I am using IgniteDataStreamer to take records and process them via the
>>>> receiver in IgniteDataStreamer and as a result of the process, for some
>>>> records, I would like to send messages into another Kafka Topic inside
>>>> the
>>>> receiver. Could you please advice a good way to do that or maybe this is
>>>> already possible by using some common library in ignite?
>>>>
>>>> Currently i am using ignite messaging to send records into the
>>>> locallisteners of the datastreamer.But i am not sure if that is a good
>>>> idea
>>>> once i have huge traffic.
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-ignite-users.705
>>>> 18.x6.nabble.com/Best-way-to-send-records-to-Kafka-from-Data
>>>> Streamer-Receiver-tp13291.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>
>
Re: Best way to send records to Kafka from DataStreamer Receiver
Posted by Nikolai Tikhonov <nt...@apache.org>.
I'm glad to hear that you were able to solve this problem yourself! If you
will face with some problems feel free to ask.
Thanks,
Nikolay
On Mon, Jun 5, 2017 at 3:28 PM, fatih tekin <fa...@gmail.com> wrote:
> Hi Nikolai,
> Thanks for responding, That is good idea but we are having some metric for
> the kafka messages and i don't want to put the our common jar into the
> classpath of the ignite. For now, I am sending all the messages in one go
> to as Json String. So that also reduces the network round as well.
>
> On Mon, Jun 5, 2017 at 1:58 PM, Nikolai Tikhonov <nt...@apache.org>
> wrote:
>
>> Hi,
>>
>> You can avoid sending message to local lister. From datastreamer receiver
>> you can submit a closure (which will send a message to kafka) to another
>> thread pool.
>>
>> On Wed, May 31, 2017 at 4:13 PM, fatality <fa...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> I am using IgniteDataStreamer to take records and process them via the
>>> receiver in IgniteDataStreamer and as a result of the process, for some
>>> records, I would like to send messages into another Kafka Topic inside
>>> the
>>> receiver. Could you please advice a good way to do that or maybe this is
>>> already possible by using some common library in ignite?
>>>
>>> Currently i am using ignite messaging to send records into the
>>> locallisteners of the datastreamer.But i am not sure if that is a good
>>> idea
>>> once i have huge traffic.
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/Best-way-to-send-records-to-Kafka-from-Data
>>> Streamer-Receiver-tp13291.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>
Re: Best way to send records to Kafka from DataStreamer Receiver
Posted by fatih tekin <fa...@gmail.com>.
Hi Nikolai,
Thanks for responding, That is good idea but we are having some metric for
the kafka messages and i don't want to put the our common jar into the
classpath of the ignite. For now, I am sending all the messages in one go
to as Json String. So that also reduces the network round as well.
On Mon, Jun 5, 2017 at 1:58 PM, Nikolai Tikhonov <nt...@apache.org>
wrote:
> Hi,
>
> You can avoid sending message to local lister. From datastreamer receiver
> you can submit a closure (which will send a message to kafka) to another
> thread pool.
>
> On Wed, May 31, 2017 at 4:13 PM, fatality <fa...@gmail.com> wrote:
>
>> Hi
>>
>> I am using IgniteDataStreamer to take records and process them via the
>> receiver in IgniteDataStreamer and as a result of the process, for some
>> records, I would like to send messages into another Kafka Topic inside the
>> receiver. Could you please advice a good way to do that or maybe this is
>> already possible by using some common library in ignite?
>>
>> Currently i am using ignite messaging to send records into the
>> locallisteners of the datastreamer.But i am not sure if that is a good
>> idea
>> once i have huge traffic.
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Best-way-to-send-records-to-Kafka-from-Dat
>> aStreamer-Receiver-tp13291.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>