You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by si...@barclays.com on 2022/02/03 05:55:19 UTC
Client Node disconnects running a long Running Compute Job
We have a ignite cluster of 1 node and a thick client connecting to it.
We have a compute job which queries about 400k records and process them and it is a long running job. We observe that the client node is getting disconnected after sometime with below error although I see that the job is still running on the ignite node. Is there a better way to handle this scenario?
2022-02-03 00:37:13,808|jvm-pause-detector-worker|||WARN |org.apache.ignite.internal.IgniteKernal.warning.290| - Possible too long JVM pause: 887 milliseconds.
2022-02-03 00:37:17,006|jvm-pause-detector-worker|||WARN |org.apache.ignite.internal.IgniteKernal.warning.290| - Possible too long JVM pause: 1049 milliseconds.
2022-02-03 00:37:18,706|jvm-pause-detector-worker|||WARN |org.apache.ignite.internal.IgniteKernal.warning.290| - Possible too long JVM pause: 758 milliseconds.
2022-02-03 00:37:21,070|jvm-pause-detector-worker|||WARN |org.apache.ignite.internal.IgniteKernal.warning.290| - Possible too long JVM pause: 812 milliseconds.
2022-02-03 00:40:00,014|scheduling-1|||INFO |com.barclays.blade.user.UserCache.refreshIncrementalCache.284| - Incremental User Cache started
2022-02-03 00:40:29,350|grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%|||WARN |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| - Communication SPI session write timed out (consider increasing 'socketWriteTimeout' configuration property) [remoteAddr=/127.0.0.1:47100, writeTimeout=2000]
2022-02-03 00:40:36,982|grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - Accepted incoming communication connection [locAddr=/127.0.0.1:47101, rmtAddr=/127.0.0.1:64918]
2022-02-03 00:40:37,068|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=null, node addrs=[/127.0.0.1:47100, /0:0:0:0:0:0:0:1:47100, .com/10.182.15.255:47100], duration=7706ms]
2022-02-03 00:40:39,116|grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%|||WARN |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| - Communication SPI session write timed out (consider increasing 'socketWriteTimeout' configuration property) [remoteAddr=/127.0.0.1:64918, writeTimeout=2000]
2022-02-03 00:40:39,417|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=null, node addrs=[/127.0.0.1:47100, /0:0:0:0:0:0:0:1:47100, .com/10.182.15.255:47100], duration=262ms]
2022-02-03 00:40:40,512|grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - Established outgoing communication connection [locAddr=/127.0.0.1:64924, rmtAddr=/127.0.0.1:47100]
2022-02-03 00:40:40,520|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=3, bytesRcvd=0, bytesSent=0, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866840512, hashCode=2072057817, interrupted=false, runner=grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=9, queueLimit=4096, reserveCnt=10, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=9, queueLimit=4096, reserveCnt=10, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=org.apache.ignite.internal.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64924, rmtAddr=/127.0.0.1:47100, createTime=1643866840512, closeTime=0, bytesSent=0, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1643866840512, lastSndTime=1643866840512, lastRcvTime=1643866840512, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]], super=GridAbstractCommunicationClient [lastUsed=1643866840512, closed=false, connIdx=0]], duration=901ms]
2022-02-03 00:40:43,062|grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%|||ERROR|org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.error.310| - Failed to process selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=3, bytesRcvd=0, bytesSent=131072, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866842054, hashCode=2072057817, interrupted=false, runner=grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=9, queueLimit=4096, reserveCnt=10, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=9, queueLimit=4096, reserveCnt=10, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64924, rmtAddr=/127.0.0.1:47100, createTime=1643866840512, closeTime=0, bytesSent=131072, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1643866840512, lastSndTime=1643866840522, lastRcvTime=1643866840512, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]]]
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:378)
at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1337)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2494)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2259)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1896)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
2022-02-03 00:40:43,083|grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%|||WARN |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| - Client disconnected abruptly due to network connection loss or because the connection was left open on application shutdown. [cls=class o.a.i.i.util.nio.GridNioException, msg=An existing connection was forcibly closed by the remote host]
2022-02-03 00:40:43,159|grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - Established outgoing communication connection [locAddr=/127.0.0.1:64925, rmtAddr=/127.0.0.1:47100]
2022-02-03 00:40:45,377|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=null, node addrs=[/127.0.0.1:47100, /0:0:0:0:0:0:0:1:47100, .com/10.182.15.255:47100], duration=207ms]
2022-02-03 00:40:46,573|grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - Established outgoing communication connection [locAddr=/127.0.0.1:64928, rmtAddr=/127.0.0.1:47100]
2022-02-03 00:40:46,580|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=160750, bytesSent=6606194144, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866846575, hashCode=309748816, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=13, queueLimit=4096, reserveCnt=14, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=13, queueLimit=4096, reserveCnt=14, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=org.apache.ignite.internal.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64928, rmtAddr=/127.0.0.1:47100, createTime=1643866846565, closeTime=0, bytesSent=0, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1643866846565, lastSndTime=1643866846575, lastRcvTime=1643866846565, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]], super=GridAbstractCommunicationClient [lastUsed=1643866846575, closed=false, connIdx=0]], duration=995ms]
2022-02-03 00:40:49,403|grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%|||ERROR|org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.error.310| - Failed to process selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=160750, bytesSent=6606325216, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866848398, hashCode=309748816, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=13, queueLimit=4096, reserveCnt=14, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=13, queueLimit=4096, reserveCnt=14, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64928, rmtAddr=/127.0.0.1:47100, createTime=1643866846565, closeTime=0, bytesSent=131072, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1643866846565, lastSndTime=1643866846575, lastRcvTime=1643866846565, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]]]
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:378)
at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1337)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2494)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2259)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1896)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
2022-02-03 00:40:49,414|grid-nio-worker-tcp-comm-1-#24%TcpCommunicationSpi%|||WARN |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| - Client disconnected abruptly due to network connection loss or because the connection was left open on application shutdown. [cls=class o.a.i.i.util.nio.GridNioException, msg=An existing connection was forcibly closed by the remote host]
2022-02-03 00:40:49,562|grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - Established outgoing communication connection [locAddr=/127.0.0.1:64930, rmtAddr=/127.0.0.1:47100]
2022-02-03 00:40:49,569|tcp-comm-worker-#1-#28|||INFO |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| - TCP client created [client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=42, bytesSent=163858, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866849569, hashCode=939500709, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=14, queueLimit=4096, reserveCnt=15, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=14, queueLimit=4096, reserveCnt=15, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=org.apache.ignite.internal.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64930, rmtAddr=/127.0.0.1:47100, createTime=1643866849559, closeTime=0, bytesSent=0, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1643866849559, lastSndTime=1643866849569, lastRcvTime=1643866849559, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]], super=GridAbstractCommunicationClient [lastUsed=1643866849569, closed=false, connIdx=0]], duration=152ms]
2022-02-03 00:40:51,984|grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%|||ERROR|org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.error.310| - Failed to process selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=42, bytesSent=294930, bytesRcvd0=0, bytesSent0=131072, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=TcpCommunicationSpi, finished=false, heartbeatTs=1643866850980, hashCode=939500709, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=14, queueLimit=4096, reserveCnt=15, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=522, resendCnt=6, rcvCnt=50, sentCnt=529, reserved=true, lastAck=32, nodeLeft=false, node=TcpDiscoveryNode [id=07185e08-a229-4dfc-9ff1-d41c4e3eaf7c, consistentId=local, addrs=ArrayList [0:0:0:0:0:0:0:1, 10.182.15.255, 127.0.0.1], sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, .com/10.182.15.255:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1643866455252, loc=false, ver=2.10.0#20210310-sha1:bc24f6ba, isClient=false], connected=false, connectCnt=14, queueLimit=4096, reserveCnt=15, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/127.0.0.1:64930, rmtAddr=/127.0.0.1:47100, createTime=1643866849559, closeTime=0, bytesSent=131072, bytesRcvd=0, bytesSent0=131072, bytesRcvd0=0, sndSchedTime=1643866849559, lastSndTime=1643866849569, lastRcvTime=1643866849559, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@955adf5, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]]]
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
This message is for information purposes only. It is not a recommendation, advice, offer or solicitation to buy or sell a product or service, nor an official confirmation of any transaction. It is directed at persons who are professionals and is intended for the recipient(s) only. It is not directed at retail customers. This message is subject to the terms at: https://www.cib.barclays/disclosures/web-and-email-disclaimer.html.
For important disclosures, please see: https://www.cib.barclays/disclosures/sales-and-trading-disclaimer.html regarding marketing commentary from Barclays Sales and/or Trading desks, who are active market participants; https://www.cib.barclays/disclosures/barclays-global-markets-disclosures.html regarding our standard terms for Barclays Corporate and Investment Bank where we trade with you in principal-to-principal wholesale markets transactions; and in respect to Barclays Research, including disclosures relating to specific issuers, see: http://publicresearch.barclays.com.
__________________________________________________________________________________
If you are incorporated or operating in Australia, read these important disclosures: https://www.cib.barclays/disclosures/important-disclosures-asia-pacific.html.
__________________________________________________________________________________
For more details about how we use personal information, see our privacy notice: https://www.cib.barclays/disclosures/personal-information-use.html.
__________________________________________________________________________________