You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Shubham Shirur <sh...@gmail.com> on 2021/08/19 04:40:03 UTC
Apache Ignite Sink Connector
Hi,
I did not find any specific documentation on Apache Ignite Sink Connector.
I am using a kafka ignite sink connector and want to push some kafka topic
data in ignite where kafka and ignite are on remote nodes. My connector
should ideally run on a remote node from the ignite node.
How can I achieve this?
What configurations I need to pass in the spring xml file which I pass as a
parameter in connector properties?
What configurations I need to pass in the spring xml file of the ignite
server node?
Thanks,
Shubham
Re: Apache Ignite Sink Connector
Posted by Saikat Maitra <sa...@gmail.com>.
Hi,
Can you please check these following performance and troubleshooting tips
and share if these are helpful in your case?
persistence tuning
https://ignite.apache.org/docs/latest/perf-and-troubleshooting/persistence-tuning
Mem and JVM tuning
https://ignite.apache.org/docs/latest/perf-and-troubleshooting/memory-tuning
Regards,
Saikat
On Thu, Sep 30, 2021 at 9:03 AM Shubham Shirur <sh...@gmail.com>
wrote:
> Hi,
>
>
> My ignite cluster contains 3 nodes. And I am using java thick client for
> processing the data present in cache. But when RAM is at its peak and
> completely filled. Cluster server starts getting down.
>
> Can you help me with this? How can I tune it ?
>
> Also, I am using Ignite's native persistence performance for
> transformation is not up to the mark.
>
> Can any configuration help me here ?
>
> Thanks & Regards,
> Shubham
>
> On Wed, Sep 22, 2021, 6:36 AM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Yes, it makes sense to connect the kafka connector to an existing cluster
>> of Ignite nodes.
>>
>> Are you facing any issues with it?
>>
>> Here is the docs for clustering and discovery process in Ignite
>> https://ignite.apache.org/docs/latest/clustering/clustering
>>
>> Let us know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Mon, Sep 20, 2021 at 4:09 AM Shubham Shirur <sh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> With parallelism I mean number for connectors in parallel.
>>>
>>> Even if group id is same, it will work like any standalone application.
>>> If node get down due to some reason, connector running on node will be down
>>> as well. In distributed mode, JVMs are shared across connect nodes which
>>> makes it more available.
>>>
>>> Thanks and Regards,
>>> Shubham
>>>
>>> On Mon, Sep 13, 2021, 1:23 AM Saikat Maitra <sa...@gmail.com>
>>> wrote:
>>>
>>>> Are you looking for a parallel Ignite sink connector that can connect
>>>> to kafka topic?
>>>>
>>>> My thoughts are you should be able to do it as long as your group_id is
>>>> same like mentioned here
>>>> https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a
>>>>
>>>> I may have misunderstood the question as well.
>>>>
>>>> Regards,
>>>> Saikat
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <sh...@gmail.com>
>>>> wrote:
>>>>
>>>>> Yes, I agree with you. Basically kafka connect works in two different
>>>>> ways i.e standalone and distributed mode.
>>>>>
>>>>> In standard mode ignite sink connectors run very smoothly. So if I
>>>>> want to create multiple topic cache data transmission, I need to setup same
>>>>> number of workers and connectors which is not recommended in production
>>>>> because its standalone, if connect node goes down connector will stop.
>>>>>
>>>>> But in distributed mode single worker is running across nodes and
>>>>> rebalancing happens for connectors tasks if any node goes down. In this
>>>>> case I am able to create just 1 ignite connector at a time. No parallelism.
>>>>>
>>>>> Hope this gives better clarity.
>>>>>
>>>>> Thanks and regards,
>>>>> Shubham
>>>>>
>>>>> On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Can you please elaborate on distributed mode?
>>>>>>
>>>>>> You can always connect multiple client to different kafka topic and
>>>>>> write data to cluster of Ignite nodes.
>>>>>>
>>>>>> Regards,
>>>>>> Saikat
>>>>>>
>>>>>> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <
>>>>>> shirurshubham@gmail.com> wrote:
>>>>>>
>>>>>>> Hey,
>>>>>>>
>>>>>>> Thanks for replying. I have gone through the documentation and could
>>>>>>> setup and run connectors in standalone mode as described there in docs. But
>>>>>>> I want to run it in distributed mode, can you help me with that?
>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>>> Shubham
>>>>>>>
>>>>>>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Shubham,
>>>>>>>>
>>>>>>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>>>>>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>>>>>>
>>>>>>>> Please let us know if you have any questions.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Saikat
>>>>>>>>
>>>>>>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <
>>>>>>>> shirurshubham@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I did not find any specific documentation on Apache Ignite Sink
>>>>>>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>>>>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>>>>>>> connector should ideally run on a remote node from the ignite node.
>>>>>>>>>
>>>>>>>>> How can I achieve this?
>>>>>>>>> What configurations I need to pass in the spring xml file which I
>>>>>>>>> pass as a parameter in connector properties?
>>>>>>>>> What configurations I need to pass in the spring xml file of the
>>>>>>>>> ignite server node?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Shubham
>>>>>>>>>
>>>>>>>>
Re: Apache Ignite Sink Connector
Posted by Shubham Shirur <sh...@gmail.com>.
Hi,
My ignite cluster contains 3 nodes. And I am using java thick client for
processing the data present in cache. But when RAM is at its peak and
completely filled. Cluster server starts getting down.
Can you help me with this? How can I tune it ?
Also, I am using Ignite's native persistence performance for transformation
is not up to the mark.
Can any configuration help me here ?
Thanks & Regards,
Shubham
On Wed, Sep 22, 2021, 6:36 AM Saikat Maitra <sa...@gmail.com> wrote:
> Hi,
>
> Yes, it makes sense to connect the kafka connector to an existing cluster
> of Ignite nodes.
>
> Are you facing any issues with it?
>
> Here is the docs for clustering and discovery process in Ignite
> https://ignite.apache.org/docs/latest/clustering/clustering
>
> Let us know if you have any questions.
>
> Regards,
> Saikat
>
> On Mon, Sep 20, 2021 at 4:09 AM Shubham Shirur <sh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> With parallelism I mean number for connectors in parallel.
>>
>> Even if group id is same, it will work like any standalone application.
>> If node get down due to some reason, connector running on node will be down
>> as well. In distributed mode, JVMs are shared across connect nodes which
>> makes it more available.
>>
>> Thanks and Regards,
>> Shubham
>>
>> On Mon, Sep 13, 2021, 1:23 AM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>>> Are you looking for a parallel Ignite sink connector that can connect to
>>> kafka topic?
>>>
>>> My thoughts are you should be able to do it as long as your group_id is
>>> same like mentioned here
>>> https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a
>>>
>>> I may have misunderstood the question as well.
>>>
>>> Regards,
>>> Saikat
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <sh...@gmail.com>
>>> wrote:
>>>
>>>> Yes, I agree with you. Basically kafka connect works in two different
>>>> ways i.e standalone and distributed mode.
>>>>
>>>> In standard mode ignite sink connectors run very smoothly. So if I want
>>>> to create multiple topic cache data transmission, I need to setup same
>>>> number of workers and connectors which is not recommended in production
>>>> because its standalone, if connect node goes down connector will stop.
>>>>
>>>> But in distributed mode single worker is running across nodes and
>>>> rebalancing happens for connectors tasks if any node goes down. In this
>>>> case I am able to create just 1 ignite connector at a time. No parallelism.
>>>>
>>>> Hope this gives better clarity.
>>>>
>>>> Thanks and regards,
>>>> Shubham
>>>>
>>>> On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
>>>> wrote:
>>>>
>>>>> Can you please elaborate on distributed mode?
>>>>>
>>>>> You can always connect multiple client to different kafka topic and
>>>>> write data to cluster of Ignite nodes.
>>>>>
>>>>> Regards,
>>>>> Saikat
>>>>>
>>>>> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <
>>>>> shirurshubham@gmail.com> wrote:
>>>>>
>>>>>> Hey,
>>>>>>
>>>>>> Thanks for replying. I have gone through the documentation and could
>>>>>> setup and run connectors in standalone mode as described there in docs. But
>>>>>> I want to run it in distributed mode, can you help me with that?
>>>>>>
>>>>>> Thanks & Regards,
>>>>>> Shubham
>>>>>>
>>>>>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Shubham,
>>>>>>>
>>>>>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>>>>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>>>>>
>>>>>>> Please let us know if you have any questions.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Saikat
>>>>>>>
>>>>>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <
>>>>>>> shirurshubham@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I did not find any specific documentation on Apache Ignite Sink
>>>>>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>>>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>>>>>> connector should ideally run on a remote node from the ignite node.
>>>>>>>>
>>>>>>>> How can I achieve this?
>>>>>>>> What configurations I need to pass in the spring xml file which I
>>>>>>>> pass as a parameter in connector properties?
>>>>>>>> What configurations I need to pass in the spring xml file of the
>>>>>>>> ignite server node?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Shubham
>>>>>>>>
>>>>>>>
Re: Apache Ignite Sink Connector
Posted by Saikat Maitra <sa...@gmail.com>.
Hi,
Yes, it makes sense to connect the kafka connector to an existing cluster
of Ignite nodes.
Are you facing any issues with it?
Here is the docs for clustering and discovery process in Ignite
https://ignite.apache.org/docs/latest/clustering/clustering
Let us know if you have any questions.
Regards,
Saikat
On Mon, Sep 20, 2021 at 4:09 AM Shubham Shirur <sh...@gmail.com>
wrote:
> Hi,
>
> With parallelism I mean number for connectors in parallel.
>
> Even if group id is same, it will work like any standalone application. If
> node get down due to some reason, connector running on node will be down as
> well. In distributed mode, JVMs are shared across connect nodes which makes
> it more available.
>
> Thanks and Regards,
> Shubham
>
> On Mon, Sep 13, 2021, 1:23 AM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Are you looking for a parallel Ignite sink connector that can connect to
>> kafka topic?
>>
>> My thoughts are you should be able to do it as long as your group_id is
>> same like mentioned here
>> https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a
>>
>> I may have misunderstood the question as well.
>>
>> Regards,
>> Saikat
>>
>>
>>
>>
>>
>> On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <sh...@gmail.com>
>> wrote:
>>
>>> Yes, I agree with you. Basically kafka connect works in two different
>>> ways i.e standalone and distributed mode.
>>>
>>> In standard mode ignite sink connectors run very smoothly. So if I want
>>> to create multiple topic cache data transmission, I need to setup same
>>> number of workers and connectors which is not recommended in production
>>> because its standalone, if connect node goes down connector will stop.
>>>
>>> But in distributed mode single worker is running across nodes and
>>> rebalancing happens for connectors tasks if any node goes down. In this
>>> case I am able to create just 1 ignite connector at a time. No parallelism.
>>>
>>> Hope this gives better clarity.
>>>
>>> Thanks and regards,
>>> Shubham
>>>
>>> On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
>>> wrote:
>>>
>>>> Can you please elaborate on distributed mode?
>>>>
>>>> You can always connect multiple client to different kafka topic and
>>>> write data to cluster of Ignite nodes.
>>>>
>>>> Regards,
>>>> Saikat
>>>>
>>>> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <
>>>> shirurshubham@gmail.com> wrote:
>>>>
>>>>> Hey,
>>>>>
>>>>> Thanks for replying. I have gone through the documentation and could
>>>>> setup and run connectors in standalone mode as described there in docs. But
>>>>> I want to run it in distributed mode, can you help me with that?
>>>>>
>>>>> Thanks & Regards,
>>>>> Shubham
>>>>>
>>>>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Shubham,
>>>>>>
>>>>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>>>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>>>>
>>>>>> Please let us know if you have any questions.
>>>>>>
>>>>>> Regards,
>>>>>> Saikat
>>>>>>
>>>>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <
>>>>>> shirurshubham@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I did not find any specific documentation on Apache Ignite Sink
>>>>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>>>>> connector should ideally run on a remote node from the ignite node.
>>>>>>>
>>>>>>> How can I achieve this?
>>>>>>> What configurations I need to pass in the spring xml file which I
>>>>>>> pass as a parameter in connector properties?
>>>>>>> What configurations I need to pass in the spring xml file of the
>>>>>>> ignite server node?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Shubham
>>>>>>>
>>>>>>
Re: Apache Ignite Sink Connector
Posted by Shubham Shirur <sh...@gmail.com>.
Hi,
With parallelism I mean number for connectors in parallel.
Even if group id is same, it will work like any standalone application. If
node get down due to some reason, connector running on node will be down as
well. In distributed mode, JVMs are shared across connect nodes which makes
it more available.
Thanks and Regards,
Shubham
On Mon, Sep 13, 2021, 1:23 AM Saikat Maitra <sa...@gmail.com> wrote:
> Are you looking for a parallel Ignite sink connector that can connect to
> kafka topic?
>
> My thoughts are you should be able to do it as long as your group_id is
> same like mentioned here
> https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a
>
> I may have misunderstood the question as well.
>
> Regards,
> Saikat
>
>
>
>
>
> On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <sh...@gmail.com>
> wrote:
>
>> Yes, I agree with you. Basically kafka connect works in two different
>> ways i.e standalone and distributed mode.
>>
>> In standard mode ignite sink connectors run very smoothly. So if I want
>> to create multiple topic cache data transmission, I need to setup same
>> number of workers and connectors which is not recommended in production
>> because its standalone, if connect node goes down connector will stop.
>>
>> But in distributed mode single worker is running across nodes and
>> rebalancing happens for connectors tasks if any node goes down. In this
>> case I am able to create just 1 ignite connector at a time. No parallelism.
>>
>> Hope this gives better clarity.
>>
>> Thanks and regards,
>> Shubham
>>
>> On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>>> Can you please elaborate on distributed mode?
>>>
>>> You can always connect multiple client to different kafka topic and
>>> write data to cluster of Ignite nodes.
>>>
>>> Regards,
>>> Saikat
>>>
>>> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <sh...@gmail.com>
>>> wrote:
>>>
>>>> Hey,
>>>>
>>>> Thanks for replying. I have gone through the documentation and could
>>>> setup and run connectors in standalone mode as described there in docs. But
>>>> I want to run it in distributed mode, can you help me with that?
>>>>
>>>> Thanks & Regards,
>>>> Shubham
>>>>
>>>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Shubham,
>>>>>
>>>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>>>
>>>>> Please let us know if you have any questions.
>>>>>
>>>>> Regards,
>>>>> Saikat
>>>>>
>>>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <
>>>>> shirurshubham@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I did not find any specific documentation on Apache Ignite Sink
>>>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>>>> connector should ideally run on a remote node from the ignite node.
>>>>>>
>>>>>> How can I achieve this?
>>>>>> What configurations I need to pass in the spring xml file which I
>>>>>> pass as a parameter in connector properties?
>>>>>> What configurations I need to pass in the spring xml file of the
>>>>>> ignite server node?
>>>>>>
>>>>>> Thanks,
>>>>>> Shubham
>>>>>>
>>>>>
Re: Apache Ignite Sink Connector
Posted by Saikat Maitra <sa...@gmail.com>.
Are you looking for a parallel Ignite sink connector that can connect to
kafka topic?
My thoughts are you should be able to do it as long as your group_id is
same like mentioned here
https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a
I may have misunderstood the question as well.
Regards,
Saikat
On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <sh...@gmail.com>
wrote:
> Yes, I agree with you. Basically kafka connect works in two different ways
> i.e standalone and distributed mode.
>
> In standard mode ignite sink connectors run very smoothly. So if I want to
> create multiple topic cache data transmission, I need to setup same number
> of workers and connectors which is not recommended in production because
> its standalone, if connect node goes down connector will stop.
>
> But in distributed mode single worker is running across nodes and
> rebalancing happens for connectors tasks if any node goes down. In this
> case I am able to create just 1 ignite connector at a time. No parallelism.
>
> Hope this gives better clarity.
>
> Thanks and regards,
> Shubham
>
> On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Can you please elaborate on distributed mode?
>>
>> You can always connect multiple client to different kafka topic and write
>> data to cluster of Ignite nodes.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <sh...@gmail.com>
>> wrote:
>>
>>> Hey,
>>>
>>> Thanks for replying. I have gone through the documentation and could
>>> setup and run connectors in standalone mode as described there in docs. But
>>> I want to run it in distributed mode, can you help me with that?
>>>
>>> Thanks & Regards,
>>> Shubham
>>>
>>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>>> wrote:
>>>
>>>> Hi Shubham,
>>>>
>>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>>
>>>> Please let us know if you have any questions.
>>>>
>>>> Regards,
>>>> Saikat
>>>>
>>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <
>>>> shirurshubham@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I did not find any specific documentation on Apache Ignite Sink
>>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>>> connector should ideally run on a remote node from the ignite node.
>>>>>
>>>>> How can I achieve this?
>>>>> What configurations I need to pass in the spring xml file which I pass
>>>>> as a parameter in connector properties?
>>>>> What configurations I need to pass in the spring xml file of the
>>>>> ignite server node?
>>>>>
>>>>> Thanks,
>>>>> Shubham
>>>>>
>>>>
Re: Apache Ignite Sink Connector
Posted by Shubham Shirur <sh...@gmail.com>.
Yes, I agree with you. Basically kafka connect works in two different ways
i.e standalone and distributed mode.
In standard mode ignite sink connectors run very smoothly. So if I want to
create multiple topic cache data transmission, I need to setup same number
of workers and connectors which is not recommended in production because
its standalone, if connect node goes down connector will stop.
But in distributed mode single worker is running across nodes and
rebalancing happens for connectors tasks if any node goes down. In this
case I am able to create just 1 ignite connector at a time. No parallelism.
Hope this gives better clarity.
Thanks and regards,
Shubham
On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Can you please elaborate on distributed mode?
>
> You can always connect multiple client to different kafka topic and write
> data to cluster of Ignite nodes.
>
> Regards,
> Saikat
>
> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <sh...@gmail.com>
> wrote:
>
>> Hey,
>>
>> Thanks for replying. I have gone through the documentation and could
>> setup and run connectors in standalone mode as described there in docs. But
>> I want to run it in distributed mode, can you help me with that?
>>
>> Thanks & Regards,
>> Shubham
>>
>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>>> Hi Shubham,
>>>
>>> Here are the documents for Apache Ignite Sink Connector using Kafka
>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>>
>>> Please let us know if you have any questions.
>>>
>>> Regards,
>>> Saikat
>>>
>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <sh...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I did not find any specific documentation on Apache Ignite Sink
>>>> Connector. I am using a kafka ignite sink connector and want to push some
>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>>> connector should ideally run on a remote node from the ignite node.
>>>>
>>>> How can I achieve this?
>>>> What configurations I need to pass in the spring xml file which I pass
>>>> as a parameter in connector properties?
>>>> What configurations I need to pass in the spring xml file of the ignite
>>>> server node?
>>>>
>>>> Thanks,
>>>> Shubham
>>>>
>>>
Re: Apache Ignite Sink Connector
Posted by Saikat Maitra <sa...@gmail.com>.
Can you please elaborate on distributed mode?
You can always connect multiple client to different kafka topic and write
data to cluster of Ignite nodes.
Regards,
Saikat
On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <sh...@gmail.com>
wrote:
> Hey,
>
> Thanks for replying. I have gone through the documentation and could setup
> and run connectors in standalone mode as described there in docs. But I
> want to run it in distributed mode, can you help me with that?
>
> Thanks & Regards,
> Shubham
>
> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Shubham,
>>
>> Here are the documents for Apache Ignite Sink Connector using Kafka
>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>>
>> Please let us know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <sh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I did not find any specific documentation on Apache Ignite Sink
>>> Connector. I am using a kafka ignite sink connector and want to push some
>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>>> connector should ideally run on a remote node from the ignite node.
>>>
>>> How can I achieve this?
>>> What configurations I need to pass in the spring xml file which I pass
>>> as a parameter in connector properties?
>>> What configurations I need to pass in the spring xml file of the ignite
>>> server node?
>>>
>>> Thanks,
>>> Shubham
>>>
>>
Re: Apache Ignite Sink Connector
Posted by Shubham Shirur <sh...@gmail.com>.
Hey,
Thanks for replying. I have gone through the documentation and could setup
and run connectors in standalone mode as described there in docs. But I
want to run it in distributed mode, can you help me with that?
Thanks & Regards,
Shubham
On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <sa...@gmail.com> wrote:
> Hi Shubham,
>
> Here are the documents for Apache Ignite Sink Connector using Kafka
> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
>
> Please let us know if you have any questions.
>
> Regards,
> Saikat
>
> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <sh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I did not find any specific documentation on Apache Ignite Sink
>> Connector. I am using a kafka ignite sink connector and want to push some
>> kafka topic data in ignite where kafka and ignite are on remote nodes. My
>> connector should ideally run on a remote node from the ignite node.
>>
>> How can I achieve this?
>> What configurations I need to pass in the spring xml file which I pass as
>> a parameter in connector properties?
>> What configurations I need to pass in the spring xml file of the ignite
>> server node?
>>
>> Thanks,
>> Shubham
>>
>
Re: Apache Ignite Sink Connector
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Shubham,
Here are the documents for Apache Ignite Sink Connector using Kafka
https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext
Please let us know if you have any questions.
Regards,
Saikat
On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <sh...@gmail.com>
wrote:
> Hi,
>
> I did not find any specific documentation on Apache Ignite Sink Connector.
> I am using a kafka ignite sink connector and want to push some kafka topic
> data in ignite where kafka and ignite are on remote nodes. My connector
> should ideally run on a remote node from the ignite node.
>
> How can I achieve this?
> What configurations I need to pass in the spring xml file which I pass as
> a parameter in connector properties?
> What configurations I need to pass in the spring xml file of the ignite
> server node?
>
> Thanks,
> Shubham
>