You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by shubhamshirur <sh...@gmail.com> on 2021/05/17 13:01:46 UTC
kafka-connect and apache-ignite
Is it compulsive to run sink-connector and ignite-cluster on same node? If
not what changes I should make in my ignite cfg file. Thank you
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: kafka-connect and apache-ignite
Posted by shubhamshirur <sh...@gmail.com>.
Where I should mention which IPs can you please tell me ? I have kafka server
on one of AWS instance and ignite on another instance. Can you help?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="peerClassLoadingEnabled" value="true"/>
<property name="cacheConfiguration">
<list>
<bean
class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="myCacheTarget"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
</bean>
</list>
</property>
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="localPort" value="49500"/>
<property name="localPortRange" value="20"/>
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>127.0.0.1:49500..49520</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="localPort" value="49100"/>
</bean>
</property>
</bean>
</beans>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: kafka-connect and apache-ignite
Posted by akorensh <al...@gmail.com>.
The Ignite servers should be set in the discovery section.
<property name="addresses">
<list>
<value>127.0.0.1:49500..49520</value>
</list>
</property>
see: https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery
The kafka servers should be set via bootstrap.servers property of the kafka
worker.
see:
https://ignite.apache.org/docs/latest/extensions-and-integrations/streaming/kafka-streamer#setting-up-and-running
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: kafka-connect and apache-ignite
Posted by shubhamshirur <sh...@gmail.com>.
Where I should mention which IPs can you please tell me ? I have kafka server
on one of AWS instance and ignite on another instance. Can you help?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="peerClassLoadingEnabled" value="true"/>
<property name="cacheConfiguration">
<list>
<bean
class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="myCacheTarget"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
</bean>
</list>
</property>
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="localPort" value="49500"/>
<property name="localPortRange" value="20"/>
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>127.0.0.1:49500..49520</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="localPort" value="49100"/>
</bean>
</property>
</bean>
</beans>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: kafka-connect and apache-ignite
Posted by akorensh <al...@gmail.com>.
Hi,
You do not need to run the sink connector and ignite cluster on the same
node.
The igniteCfg property points to the connector's ignite configuration
file.
https://ignite.apache.org/docs/latest/extensions-and-integrations/streaming/kafka-streamer
(see step 3)
Use the discovery section of the ignite config file to indicate where to
find the cluster.
See: https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery
Thanks, Alex
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/