You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Prasad Bhalerao <pr...@gmail.com> on 2019/12/09 13:11:46 UTC
Topology version changing very frequently
My application nodes are running on VMs and observing that topology version
is changing very frequently but I do not see any error/exceptions/warnings
in log files.
For ignite I have kept the log level to WARN.
Timeout settings in millis are as follows:
FailureDetectionTimeout = 120000
NetworkTimeout = 10000
ClientFailureDetectionTimeout = 120000
1) Does the topology version change when node becomes unreachable and
becomes reachable again within the failure detection time out?
2) Which time out setting does ignite use while deciding that topology
version is changed?
3) Why the node is not getting segmented in this case? Is it because the
node is becoming reachable within failure detection timeout?
4) What is the purpose of network timeout ?
*App Logs::*
*Topology snapshot [ver=18, servers=4, clients=6, CPUs=172, offheap=40.0GB,
heap=160.0GB]Topology snapshot [ver=19, servers=4, clients=5, CPUs=152,
offheap=40.0GB, heap=140.0GB]Topology snapshot [ver=20, servers=4,
clients=4, CPUs=132, offheap=40.0GB, heap=130.0GB] *
*...*
*..*
*.Topology snapshot [ver=56, servers=4, clients=4, CPUs=132,
offheap=40.0GB, heap=130.0GB]Topology snapshot [ver=57, servers=4,
clients=5, CPUs=152, offheap=40.0GB, heap=140.0GB]Topology snapshot
[ver=58, servers=4, clients=4, CPUs=132, offheap=40.0GB,
heap=130.0GB]Topology snapshot [ver=59, servers=4, clients=5, CPUs=152,
offheap=40.0GB, heap=140.0GB] *
Re: Topology version changing very frequently
Posted by Denis Magda <dm...@apache.org>.
Check the logs of all your servers nodes and client applications, you
should see why the nodes stop and start. The topology changes only if a
node leaves the cluster or joins it.
The failureDetectionTimeout is used by default for the network
communication timeouts between the nodes:
https://apacheignite.readme.io/docs/tcpip-discovery#section-failure-detection-timeout
-
Denis
On Tue, Dec 10, 2019 at 10:30 PM Prasad Bhalerao <
prasadbhalerao1983@gmail.com> wrote:
> Can you please try to answer question 1, 2 and 3?
>
> In my logs I could see that number of server are also changing. Topology
> version changes when server node becomes 3 and again becomes 4. Total
> number of server nodes are 4.
>
> Thanks,
> Prasad
>
>
> On Tue 10 Dec, 2019, 11:45 PM akurbanov <antkr.dev@gmail.com wrote:
>
>> Hello,
>>
>> Are you able to share full logs from the server and client instances?
>>
>> In short: by default clients can reconnect to the cluster after the
>> network
>> connection was disrupted by anything (network issues, gc pauses etc.).
>>
>>
>> https://apacheignite.readme.io/docs/clients-vs-servers#section-reconnecting-a-client
>>
>> Server drops client node from topology once failure detection timeout is
>> reached, if you want a client to be stopped and segmented in this case,
>> use
>> property clientReconnectDisabled and set it to true, the sample is on the
>> documentation page.
>>
>> Network timeout defines the timeout for different network operations,
>> among
>> the usages are client reconnect attempt timeout, connection establishing
>> timeout on join attempt and sending messages.
>>
>> Best regards,
>> Anton
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: Topology version changing very frequently
Posted by Prasad Bhalerao <pr...@gmail.com>.
Can you please try to answer question 1, 2 and 3?
In my logs I could see that number of server are also changing. Topology
version changes when server node becomes 3 and again becomes 4. Total
number of server nodes are 4.
Thanks,
Prasad
On Tue 10 Dec, 2019, 11:45 PM akurbanov <antkr.dev@gmail.com wrote:
> Hello,
>
> Are you able to share full logs from the server and client instances?
>
> In short: by default clients can reconnect to the cluster after the network
> connection was disrupted by anything (network issues, gc pauses etc.).
>
>
> https://apacheignite.readme.io/docs/clients-vs-servers#section-reconnecting-a-client
>
> Server drops client node from topology once failure detection timeout is
> reached, if you want a client to be stopped and segmented in this case, use
> property clientReconnectDisabled and set it to true, the sample is on the
> documentation page.
>
> Network timeout defines the timeout for different network operations, among
> the usages are client reconnect attempt timeout, connection establishing
> timeout on join attempt and sending messages.
>
> Best regards,
> Anton
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Topology version changing very frequently
Posted by akurbanov <an...@gmail.com>.
Hello,
Are you able to share full logs from the server and client instances?
In short: by default clients can reconnect to the cluster after the network
connection was disrupted by anything (network issues, gc pauses etc.).
https://apacheignite.readme.io/docs/clients-vs-servers#section-reconnecting-a-client
Server drops client node from topology once failure detection timeout is
reached, if you want a client to be stopped and segmented in this case, use
property clientReconnectDisabled and set it to true, the sample is on the
documentation page.
Network timeout defines the timeout for different network operations, among
the usages are client reconnect attempt timeout, connection establishing
timeout on join attempt and sending messages.
Best regards,
Anton
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/