You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by adipro <ad...@zohocorp.com> on 2020/09/10 07:17:36 UTC

DC replication via Kafka Connect for Apache Ignite

We are trying to implement the replication of data between two Ignite data
clusters.

For this purpose, we are using Kafka Connect.

We have followed the things mentioned in this document ->
https://dzone.com/articles/linking-apache-ignite-and-apache-kafka-for-highly

Everything is working fine till we use one cache and PUT operation.

But when I use the same for REMOVED operation, in the consumer thread of the
connector, I can see the CacheEvent record, but the data is not being
removed from the Sink Cluster nodes.

When checked the remove functionality is missing in IgniteSinkTask class.

Is IgniteSink connector only for injecting data? The data record type at
receiving end is of type CacheEvent. Based on operation it should put or
remove right? As of now only PUT is working.

Can someone please help with this case?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: DC replication via Kafka Connect for Apache Ignite

Posted by zxcware <zx...@gmail.com>.
What's current kafka version supported in Kafka Connect? From
https://www.gridgain.com/docs/8.8.2/integrations/kafka/kc-ggvai, it said the
IgniteSinkConnector just support kafka version 0.1, which is quite old.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: DC replication via Kafka Connect for Apache Ignite

Posted by ibelyakov <ig...@gmail.com>.
Ignite Kafka sink connector doesn't support removals processing. I've created
a ticket to implement this improvement:
https://issues.apache.org/jira/browse/IGNITE-13442

But, as I know, such functionality is implemented in GridGain Kafka
Connector, you can find more details regarding it and data replication,
here:
https://www.gridgain.com/docs/latest/integrations/kafka/kc-ex-replication

Igor



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/