You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Anatolii Botov (JIRA)" <ji...@apache.org> on 2019/02/06 07:17:00 UTC
[jira] [Updated] (IGNITE-11222) TcpDiscoverySpi stops listen port
on network timeout
[ https://issues.apache.org/jira/browse/IGNITE-11222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anatolii Botov updated IGNITE-11222:
------------------------------------
Description:
I've created cluster from 2 nodes.
I've configured them as follow:
{color:#000080}val {color}config = {color:#000080}new {color}IgniteConfiguration()
{color:#000080}val {color}discoverySpi = {color:#000080}new {color}TcpDiscoverySpi
discoverySpi.setLocalAddress(discoveryAddress.getHostString)
discoverySpi.setLocalPort(discoveryAddress.getPort)
{color:#000080}val {color}ipFinder = {color:#000080}new {color}TcpDiscoveryVmIpFinder()
ipFinder.setAddresses(ipAddresses.asJava)
discoverySpi.setIpFinder(ipFinder)
config.setDiscoverySpi(discoverySpi)
{color:#000080}val {color}commSpi = {color:#000080}new {color}TcpCommunicationSpi()
commSpi.setLocalAddress(communicationAddress.getHostString)
commSpi.setLocalPort(communicationAddress.getPort)
commSpi.setSlowClientQueueLimit(igniteConf.getInt({color:#008000}"slow-client-queue-limit"{color}).getOrElse({color:#0000ff}1000{color}))
commSpi.setMessageQueueLimit(
igniteConf.getInt({color:#008000}"message-queue-limit"{color}).getOrElse(TcpCommunicationSpi.{color:#660e7a}DFLT_MSG_QUEUE_LIMIT{color})
)
config.setCommunicationSpi(commSpi)
config.setSegmentationPolicy(SegmentationPolicy.{color:#660e7a}NOOP{color})
* For first node(10.84.13.131) values are:
{color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
{color:#000080}val {color}communicationAddress = 0.0.0.0:47600
{color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"127.0.0.1"{color})
* For second node(10.84.0.120) values are:
{color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
{color:#000080}val {color}communicationAddress = 0.0.0.0:47600
{color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"10.84.0.120:47500"{color})
After i start first node and the second second connects to first and everything is OK. Then i block traffic from first node to second:
iptables -A OUTPUT -d 10.84.0.120 -j DROP
There is 2 cases:
# if traffic is blocked more than default timeout in ignite config, everything works as expected: split brain.
# But if i remove blocking before timeout is almost expired. Then first node closes socket on port 47500 and becomes unreachable for further reconnection attempts from first node.
was:
I've created cluster from 2 nodes.
I've configured them as follow:
{color:#000080}val {color}config = {color:#000080}new {color}IgniteConfiguration()
{color:#000080}val {color}discoverySpi = {color:#000080}new {color}TcpDiscoverySpi
discoverySpi.setLocalAddress(discoveryAddress.getHostString)
discoverySpi.setLocalPort(discoveryAddress.getPort)
{color:#000080}val {color}ipFinder = {color:#000080}new {color}TcpDiscoveryVmIpFinder()
ipFinder.setAddresses(ipAddresses.asJava)
discoverySpi.setIpFinder(ipFinder)
config.setDiscoverySpi(discoverySpi)
{color:#000080}val {color}commSpi = {color:#000080}new {color}TcpCommunicationSpi()
commSpi.setLocalAddress(communicationAddress.getHostString)
commSpi.setLocalPort(communicationAddress.getPort)
commSpi.setSlowClientQueueLimit(igniteConf.getInt({color:#008000}"slow-client-queue-limit"{color}).getOrElse({color:#0000ff}1000{color}))
commSpi.setMessageQueueLimit(
igniteConf.getInt({color:#008000}"message-queue-limit"{color}).getOrElse(TcpCommunicationSpi.{color:#660e7a}DFLT_MSG_QUEUE_LIMIT{color})
)
config.setCommunicationSpi(commSpi)
config.setSegmentationPolicy(SegmentationPolicy.{color:#660e7a}NOOP{color})
For first node(10.84.13.131) values are:
{color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
{color:#000080}val {color}communicationAddress = 0.0.0.0:47600
{color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"127.0.0.1"{color})
For second node(10.84.0.120) values are:
{color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
{color:#000080}val {color}communicationAddress = 0.0.0.0:47600
{color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"10.84.0.120:47500"{color})
After i start first node and the second second connects to first and everything is OK. Then i block traffic from first node to second:
iptables -A OUTPUT -d 10.84.0.120 -j DROP
There is 2 cases:
# if traffic is blocked more than default timeout in ignite config, everything works as expected: split brain.
# But if i remove blocking before timeout is almost expired. Then first node closes socket on port 47500 and becomes unreachable for further reconnection attempts from first node.
> TcpDiscoverySpi stops listen port on network timeout
> ----------------------------------------------------
>
> Key: IGNITE-11222
> URL: https://issues.apache.org/jira/browse/IGNITE-11222
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.7
> Environment: OS: Ubuntu 18.04, Centos 7
> Ignite runs as part of application through Ignition.start()
>
>
> Reporter: Anatolii Botov
> Priority: Major
> Attachments: ignite_logs.zip
>
>
> I've created cluster from 2 nodes.
> I've configured them as follow:
> {color:#000080}val {color}config = {color:#000080}new {color}IgniteConfiguration()
> {color:#000080}val {color}discoverySpi = {color:#000080}new {color}TcpDiscoverySpi
> discoverySpi.setLocalAddress(discoveryAddress.getHostString)
> discoverySpi.setLocalPort(discoveryAddress.getPort)
> {color:#000080}val {color}ipFinder = {color:#000080}new {color}TcpDiscoveryVmIpFinder()
> ipFinder.setAddresses(ipAddresses.asJava)
> discoverySpi.setIpFinder(ipFinder)
> config.setDiscoverySpi(discoverySpi)
> {color:#000080}val {color}commSpi = {color:#000080}new {color}TcpCommunicationSpi()
> commSpi.setLocalAddress(communicationAddress.getHostString)
> commSpi.setLocalPort(communicationAddress.getPort)
> commSpi.setSlowClientQueueLimit(igniteConf.getInt({color:#008000}"slow-client-queue-limit"{color}).getOrElse({color:#0000ff}1000{color}))
> commSpi.setMessageQueueLimit(
> igniteConf.getInt({color:#008000}"message-queue-limit"{color}).getOrElse(TcpCommunicationSpi.{color:#660e7a}DFLT_MSG_QUEUE_LIMIT{color})
> )
> config.setCommunicationSpi(commSpi)
> config.setSegmentationPolicy(SegmentationPolicy.{color:#660e7a}NOOP{color})
>
> * For first node(10.84.13.131) values are:
> {color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
> {color:#000080}val {color}communicationAddress = 0.0.0.0:47600
> {color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"127.0.0.1"{color})
> * For second node(10.84.0.120) values are:
> {color:#000080}val {color}discoveryAddress = 0.0.0.0:47500
> {color:#000080}val {color}communicationAddress = 0.0.0.0:47600
> {color:#000080}val {color}ipAddresses = {color:#660e7a}Seq{color}({color:#008000}"10.84.0.120:47500"{color})
>
> After i start first node and the second second connects to first and everything is OK. Then i block traffic from first node to second:
> iptables -A OUTPUT -d 10.84.0.120 -j DROP
>
> There is 2 cases:
> # if traffic is blocked more than default timeout in ignite config, everything works as expected: split brain.
> # But if i remove blocking before timeout is almost expired. Then first node closes socket on port 47500 and becomes unreachable for further reconnection attempts from first node.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)