You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by shubhamshirur <sh...@gmail.com> on 2021/05/17 13:01:46 UTC

kafka-connect and apache-ignite

Is it compulsive to run sink-connector and ignite-cluster on same node? If
not what changes I should make in my ignite cfg file. Thank you




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: kafka-connect and apache-ignite

Posted by shubhamshirur <sh...@gmail.com>.
Where I should mention which IPs can you please tell me ? I have kafka server
on one of AWS instance and ignite on another instance. Can you help?




<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="peerClassLoadingEnabled" value="true"/>
        <property name="cacheConfiguration">
            <list>
                
                <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="myCacheTarget"/>
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="1"/>
                </bean>
            </list>
        </property>
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                
                <property name="localPort" value="49500"/>

                <property name="localPortRange" value="20"/>

                <property name="ipFinder">
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:49500..49520</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
        
        <property name="communicationSpi">
            <bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="localPort" value="49100"/>
            </bean>
        </property>
    </bean>
</beans>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: kafka-connect and apache-ignite

Posted by akorensh <al...@gmail.com>.
The Ignite servers should be set in the discovery section.
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:49500..49520</value>
                            </list>
                        </property>
see:  https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery

The kafka servers should be set via bootstrap.servers property of the kafka
worker.
see:
https://ignite.apache.org/docs/latest/extensions-and-integrations/streaming/kafka-streamer#setting-up-and-running




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: kafka-connect and apache-ignite

Posted by shubhamshirur <sh...@gmail.com>.
Where I should mention which IPs can you please tell me ? I have kafka server
on one of AWS instance and ignite on another instance. Can you help?




<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="peerClassLoadingEnabled" value="true"/>
        <property name="cacheConfiguration">
            <list>
                
                <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="myCacheTarget"/>
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="1"/>
                </bean>
            </list>
        </property>
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                
                <property name="localPort" value="49500"/>

                <property name="localPortRange" value="20"/>

                <property name="ipFinder">
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:49500..49520</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
        
        <property name="communicationSpi">
            <bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="localPort" value="49100"/>
            </bean>
        </property>
    </bean>
</beans>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: kafka-connect and apache-ignite

Posted by akorensh <al...@gmail.com>.
Hi, 
   You do not need to run the sink connector and ignite cluster on the same
node.
   The igniteCfg property points to the connector's ignite configuration
file.
https://ignite.apache.org/docs/latest/extensions-and-integrations/streaming/kafka-streamer
  (see step 3)
 
   Use the discovery section of the ignite config file to indicate where to
find the cluster.
   See: https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery


Thanks, Alex 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/