You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Miles Chen <fe...@gmail.com> on 2023/06/29 14:00:04 UTC
Duplicated JMX DataTreeBean registered
Hi,
We using jmx-exporter <https://github.com/prometheus/jmx_exporter> to scrape JMX Bean from zookeeper pod(deployed on Kubernetes) and export prometheus metrics. Recently, we found a abnormal metric came out in production, as below
zookeeper_InMemoryDatatree_NodeCount{memberType=“Leader”, replicateId=“201" } 218.0
zookeeper_InMemoryDatatree_NodeCount{memberType=“Follower”, replicateId=“201" } 218.0
zookeeper_InMemoryDataTree_NodeCount metric have two sample with different memberType, but the pod should be Leader not Follower! And, we found zookeeper_InMemoryDataTree_WatchCount metric have the same issue, meanwhile the rest of metrics have not.
We tried to reproduce this by restarting a Leader Pod, make another pod to be leader, but got bad luck.
As we dived in source code, we thought it may caused by DataTreeBean duplicated, but we don’t have any evidences. I just wonder if anyone came into this issue before, or if there is chance this is a bug.
Any advices would be helpful, thanks in advanced.