You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Jeremy Torres <to...@gmail.com> on 2019/06/05 20:32:06 UTC
ClusterMetricsMXBean Stops Reporting Data
Production Setup:
Ignite Version: 2.7.0
JVM Info: Java(TM) SE Runtime Environment (build 1.8.0_77-b31);
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b31, mixed mode)
OS: Centos Linux 7 (3.10 Kernel)
3 servers nodes in Ignite cluster
3 client nodes
Issue Observed:
Monitoring of ClusterMetricsMXBean properties getReceivedMessagesCount and
getSentMessagesCount no longer reports data.
Problem Overview:
* 2 of the 3 nodes stop returning data for the 2 metrics
* The nodes appear healthy and are processing as expected. Specifically,
network traffic remains consistent
* Zabbix is used to monitor these metrics
Question:
If a server node stops reporting data, is it assumed to be unhealthy? Is
this a known issue? I've searched but couldn't find any related issues.
Thanks for your help.
Re: ClusterMetricsMXBean Stops Reporting Data
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
This is unexpected. Is it possible to create a reproducer project to
reproduce this behavior? Can you share it?
Regards,
--
Ilya Kasnacheev
пт, 7 июн. 2019 г. в 18:37, torjt <to...@gmail.com>:
> Hello. Thanks for the response. The operation returns with 0 values.
> There
> is no Zookeeper being used in this setup.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ClusterMetricsMXBean Stops Reporting Data
Posted by torjt <to...@gmail.com>.
Hello. Thanks for the response. The operation returns with 0 values. There
is no Zookeeper being used in this setup.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ClusterMetricsMXBean Stops Reporting Data
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
What do you mean by "not reporting data"? Do you get zeros/nulls? Does the
operation hang? Maybe there are exceptions?
I remember there was the following issue about metrics:
https://issues.apache.org/jira/browse/IGNITE-11352
But it's hard to trigger. Maybe, if you're using Zk discovery.
Regards,
--
Ilya Kasnacheev
ср, 5 июн. 2019 г. в 23:32, Jeremy Torres <to...@gmail.com>:
> Production Setup:
> Ignite Version: 2.7.0
> JVM Info: Java(TM) SE Runtime Environment (build 1.8.0_77-b31);
> Java HotSpot(TM) 64-Bit Server VM (build 25.77-b31, mixed mode)
> OS: Centos Linux 7 (3.10 Kernel)
> 3 servers nodes in Ignite cluster
> 3 client nodes
>
> Issue Observed:
> Monitoring of ClusterMetricsMXBean properties getReceivedMessagesCount and
> getSentMessagesCount no longer reports data.
>
> Problem Overview:
> * 2 of the 3 nodes stop returning data for the 2 metrics
> * The nodes appear healthy and are processing as expected. Specifically,
> network traffic remains consistent
> * Zabbix is used to monitor these metrics
>
> Question:
> If a server node stops reporting data, is it assumed to be unhealthy? Is
> this a known issue? I've searched but couldn't find any related issues.
>
> Thanks for your help.
>