You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Tobias König <to...@banksapi.de> on 2019/01/08 14:13:12 UTC
Cluster of two nodes with minimal port use
Hi there,
I'm trying to get an Ignite cluster consisting of two nodes to work,
that uses a minimum number of exposed ports. I'm new to Ignite, but it
is my understanding, that it should suffice to set each node to one
specific port 1. for communication and 2. for discovery. The overall
goal is to get a Docker cluster (with default bridged networking)
working without Multicast and without --net=host.
However, I'm doing preliminary tests /without/ docker and am directly
using my local machine (Node 1, IP 172.24.10.79) and a Raspberry Pi
(Node 2, IP 172.24.10.83), and I can't get the cluster to work, because
the discovery process doesn't succeed. I'm using a static IP finder in
which I point each node to its corresponding counterpart.
XML-configuration of both nodes with the aforementioned minimal use of
ports is attached inline.
If I start node 1 first and then node 2, no discovery process is
initiated in the first minutes. If I start node 2 first and then node 1,
the discovery process is initiated but not completed successfully. I'll
attach logs for the second case for both node 2 and 1.
Can somebody spot my configuration error?
Best regards and TIA,
Tobias
P.S. I was able to reproduce the error on two "regular" machines as
well, without the use of a Raspberry Pi.
_______________________________
# ignite-config-node1.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="localPort" value="3013"/>
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>127.0.0.1:3013</value>
<value>172.24.10.83:3013</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="localPort" value="3012"/>
</bean>
</property>
</bean>
</beans>
# ignite-config-node2.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="localPort" value="3013"/>
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>127.0.0.1:3013</value>
<value>172.24.10.79:3013</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="localPort" value="3012"/>
</bean>
</property>
</bean>
</beans>
Re: Cluster of two nodes with minimal port use
Posted by Tobias König <to...@banksapi.de>.
I think I can narrow it down to this error message:
[12:38:10,957][WARNING][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed
to send message to next node [msg=TcpDiscoveryNodeAddedMessage
[node=TcpDiscoveryNode [id=98c88dfd-758e-40f4-9597-eb4c4f700280,
addrs=[172.24.10.79], sockAddrs=[/172.24.10.79:3013], discPort=3013, ...
[snipped, full message below]
However, I can't find any information on /why/ the sending failed. I
even started Ignite in verbose mode.
I studied many posts by users having problems forming Ignite clusters
and from what I could gather it seems that discovery (port 3013) seems
to work, but communication (port 3012) does not. I already checked that
the ports are reachable from both hosts, of course.
Can somebody help?
______________________
[12:38:10,957][WARNING][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed
to send message to next node [msg=TcpDiscoveryNodeAddedMessage
[node=TcpDiscoveryNode [id=98c88dfd-758e-40f4-9597-eb4c4f700280,
addrs=[172.24.10.79], sockAddrs=[/172.24.10.79:3013], discPort=3013,
order=0, intOrder=6, lastExchangeTime=1547033890878, loc=false,
ver=2.7.0#20181130-sha1:256ae401, isClient=false],
dataPacket=o.a.i.spi.discovery.tcp.internal.DiscoveryDataPacket@1fdbcfd,
discardMsgId=null, discardCustomMsgId=null, top=null, clientTop=null,
gridStartTime=1547033858511, super=TcpDiscoveryAbstractMessage
[sndNodeId=null, id=98c37623861-9fe36b10-cda3-41bd-a91f-82e021edf5ec,
verifierNodeId=9fe36b10-cda3-41bd-a91f-82e021edf5ec, topVer=0,
pendingIdx=0, failedNodes=null, isClient=false]], next=TcpDiscoveryNode
[id=98c88dfd-758e-40f4-9597-eb4c4f700280, addrs=[172.24.10.79],
sockAddrs=[/172.24.10.79:3013], discPort=3013, order=0, intOrder=6,
lastExchangeTime=1547033890878, loc=false,
ver=2.7.0#20181130-sha1:256ae401, isClient=false], errMsg=Failed to send
message to next node [msg=TcpDiscoveryNodeAddedMessage
[node=TcpDiscoveryNode [id=98c88dfd-758e-40f4-9597-eb4c4f700280,
addrs=[172.24.10.79], sockAddrs=[/172.24.10.79:3013], discPort=3013,
order=0, intOrder=6, lastExchangeTime=1547033890878, loc=false,
ver=2.7.0#20181130-sha1:256ae401, isClient=false],
dataPacket=o.a.i.spi.discovery.tcp.internal.DiscoveryDataPacket@1fdbcfd,
discardMsgId=null, discardCustomMsgId=null, top=null, clientTop=null,
gridStartTime=1547033858511, super=TcpDiscoveryAbstractMessage
[sndNodeId=null, id=98c37623861-9fe36b10-cda3-41bd-a91f-82e021edf5ec,
verifierNodeId=9fe36b10-cda3-41bd-a91f-82e021edf5ec, topVer=0,
pendingIdx=0, failedNodes=null, isClient=false]], next=ClusterNode
[id=98c88dfd-758e-40f4-9597-eb4c4f700280, order=0, addr=[172.24.10.79],
daemon=false]]]
On 1/8/19 3:13 PM, Tobias König wrote:
> Hi there,
>
> I'm trying to get an Ignite cluster consisting of two nodes to work,
> that uses a minimum number of exposed ports. I'm new to Ignite, but it
> is my understanding, that it should suffice to set each node to one
> specific port 1. for communication and 2. for discovery. The overall
> goal is to get a Docker cluster (with default bridged networking)
> working without Multicast and without --net=host.
>
> However, I'm doing preliminary tests /without/ docker and am directly
> using my local machine (Node 1, IP 172.24.10.79) and a Raspberry Pi
> (Node 2, IP 172.24.10.83), and I can't get the cluster to work,
> because the discovery process doesn't succeed. I'm using a static IP
> finder in which I point each node to its corresponding counterpart.
>
> XML-configuration of both nodes with the aforementioned minimal use of
> ports is attached inline.
>
> If I start node 1 first and then node 2, no discovery process is
> initiated in the first minutes. If I start node 2 first and then node
> 1, the discovery process is initiated but not completed successfully.
> I'll attach logs for the second case for both node 2 and 1.
>
> Can somebody spot my configuration error?
>
> Best regards and TIA,
> Tobias
>
>
>
> P.S. I was able to reproduce the error on two "regular" machines as
> well, without the use of a Raspberry Pi.
>
>
> _______________________________
>
> # ignite-config-node1.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.83:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
> # ignite-config-node2.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean
> class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property
> name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.79:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
>
Re: Cluster of two nodes with minimal port use
Posted by Tobias König <to...@banksapi.de>.
Hi Stephen,
I tested your proposal, but to no avail. The discovery process is still
continuously retried, but never successful.
Best regards,
Tobias
On 1/8/19 4:07 PM, Stephen Darlington wrote:
> Try putting the same list on both nodes:
>
> <value>172.24.10.79:3013</value>
> <value>172.24.10.83:3013</value>
>
> Regards,
> Stephen
>
>> On 8 Jan 2019, at 14:13, Tobias König <to...@banksapi.de> wrote:
>>
>> Hi there,
>>
>> I'm trying to get an Ignite cluster consisting of two nodes to work, that uses a minimum number of exposed ports. I'm new to Ignite, but it is my understanding, that it should suffice to set each node to one specific port 1. for communication and 2. for discovery. The overall goal is to get a Docker cluster (with default bridged networking) working without Multicast and without --net=host.
>>
>> However, I'm doing preliminary tests /without/ docker and am directly using my local machine (Node 1, IP 172.24.10.79) and a Raspberry Pi (Node 2, IP 172.24.10.83), and I can't get the cluster to work, because the discovery process doesn't succeed. I'm using a static IP finder in which I point each node to its corresponding counterpart.
>>
>> XML-configuration of both nodes with the aforementioned minimal use of ports is attached inline.
>>
>> If I start node 1 first and then node 2, no discovery process is initiated in the first minutes. If I start node 2 first and then node 1, the discovery process is initiated but not completed successfully. I'll attach logs for the second case for both node 2 and 1.
>>
>> Can somebody spot my configuration error?
>>
>> Best regards and TIA,
>> Tobias
>>
>>
>>
>> P.S. I was able to reproduce the error on two "regular" machines as well, without the use of a Raspberry Pi.
>>
>>
>> _______________________________
>>
>> # ignite-config-node1.xml
>>
>> <beans xmlns="http://www.springframework.org/schema/beans"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="
>> http://www.springframework.org/schema/beans
>> http://www.springframework.org/schema/beans/spring-beans.xsd">
>>
>> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
>> <property name="discoverySpi">
>> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>> <property name="localPort" value="3013"/>
>> <property name="ipFinder">
>> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>> <property name="addresses">
>> <list>
>> <value>127.0.0.1:3013</value>
>> <value>172.24.10.83:3013</value>
>> </list>
>> </property>
>> </bean>
>> </property>
>> </bean>
>> </property>
>> <property name="communicationSpi">
>> <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>> <property name="localPort" value="3012"/>
>> </bean>
>> </property>
>> </bean>
>> </beans>
>>
>>
>> # ignite-config-node2.xml
>>
>> <beans xmlns="http://www.springframework.org/schema/beans"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="
>> http://www.springframework.org/schema/beans
>> http://www.springframework.org/schema/beans/spring-beans.xsd">
>>
>> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
>> <property name="discoverySpi">
>> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>> <property name="localPort" value="3013"/>
>> <property name="ipFinder">
>> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>> <property name="addresses">
>> <list>
>> <value>127.0.0.1:3013</value>
>> <value>172.24.10.79:3013</value>
>> </list>
>> </property>
>> </bean>
>> </property>
>> </bean>
>> </property>
>> <property name="communicationSpi">
>> <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>> <property name="localPort" value="3012"/>
>> </bean>
>> </property>
>> </bean>
>> </beans>
>>
>>
>>
>> <ignite-node2.log><ignite-node1.log>
>
Re: Cluster of two nodes with minimal port use
Posted by Stephen Darlington <st...@gridgain.com>.
Try putting the same list on both nodes:
<value>172.24.10.79:3013</value>
<value>172.24.10.83:3013</value>
Regards,
Stephen
> On 8 Jan 2019, at 14:13, Tobias König <to...@banksapi.de> wrote:
>
> Hi there,
>
> I'm trying to get an Ignite cluster consisting of two nodes to work, that uses a minimum number of exposed ports. I'm new to Ignite, but it is my understanding, that it should suffice to set each node to one specific port 1. for communication and 2. for discovery. The overall goal is to get a Docker cluster (with default bridged networking) working without Multicast and without --net=host.
>
> However, I'm doing preliminary tests /without/ docker and am directly using my local machine (Node 1, IP 172.24.10.79) and a Raspberry Pi (Node 2, IP 172.24.10.83), and I can't get the cluster to work, because the discovery process doesn't succeed. I'm using a static IP finder in which I point each node to its corresponding counterpart.
>
> XML-configuration of both nodes with the aforementioned minimal use of ports is attached inline.
>
> If I start node 1 first and then node 2, no discovery process is initiated in the first minutes. If I start node 2 first and then node 1, the discovery process is initiated but not completed successfully. I'll attach logs for the second case for both node 2 and 1.
>
> Can somebody spot my configuration error?
>
> Best regards and TIA,
> Tobias
>
>
>
> P.S. I was able to reproduce the error on two "regular" machines as well, without the use of a Raspberry Pi.
>
>
> _______________________________
>
> # ignite-config-node1.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.83:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
> # ignite-config-node2.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.79:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
>
> <ignite-node2.log><ignite-node1.log>
Re: Cluster of two nodes with minimal port use
Posted by Tobias König <to...@banksapi.de>.
Hi all,
nevermind, the problem was due to a malconfigured packet filter.
Cheers,
Tobias
On 1/8/19 3:13 PM, Tobias König wrote:
> Hi there,
>
> I'm trying to get an Ignite cluster consisting of two nodes to work,
> that uses a minimum number of exposed ports. I'm new to Ignite, but it
> is my understanding, that it should suffice to set each node to one
> specific port 1. for communication and 2. for discovery. The overall
> goal is to get a Docker cluster (with default bridged networking)
> working without Multicast and without --net=host.
>
> However, I'm doing preliminary tests /without/ docker and am directly
> using my local machine (Node 1, IP 172.24.10.79) and a Raspberry Pi
> (Node 2, IP 172.24.10.83), and I can't get the cluster to work,
> because the discovery process doesn't succeed. I'm using a static IP
> finder in which I point each node to its corresponding counterpart.
>
> XML-configuration of both nodes with the aforementioned minimal use of
> ports is attached inline.
>
> If I start node 1 first and then node 2, no discovery process is
> initiated in the first minutes. If I start node 2 first and then node
> 1, the discovery process is initiated but not completed successfully.
> I'll attach logs for the second case for both node 2 and 1.
>
> Can somebody spot my configuration error?
>
> Best regards and TIA,
> Tobias
>
>
>
> P.S. I was able to reproduce the error on two "regular" machines as
> well, without the use of a Raspberry Pi.
>
>
> _______________________________
>
> # ignite-config-node1.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.83:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
> # ignite-config-node2.xml
>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd">
>
> <bean
> class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="discoverySpi">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="localPort" value="3013"/>
> <property name="ipFinder">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
> <property
> name="addresses">
> <list>
> <value>127.0.0.1:3013</value>
> <value>172.24.10.79:3013</value>
> </list>
> </property>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="localPort" value="3012"/>
> </bean>
> </property>
> </bean>
> </beans>
>
>
>