You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/03/27 12:40:45 UTC
[GitHub] [helix] PauloRelex opened a new issue #918: 2 ZkClientPathMonitor
HistogramDynamicMetric using 100s of MBs each
PauloRelex opened a new issue #918: 2 ZkClientPathMonitor HistogramDynamicMetric using 100s of MBs each
URL: https://github.com/apache/helix/issues/918
My company is using Helix, Namerd, Linkerd and ZooKeeper.
During Performance Testing, Namerd exhausts its Heap _(doesn't crash but becomes functionally unusable)_ and Linkerd runs really high in memory usage.
Analysing Heap Dumps from those 2 processes, the majority of the _(seemingly excessive)_ RAM usage appears to be 2 of the 4 HistogramDynamicMetric instances under ZkClientPathMonitor. [This class?](https://github.com/apache/helix/blob/master/metrics-common/src/main/java/org/apache/helix/monitoring/mbeans/dynamicMBeans/HistogramDynamicMetric.java) It appears quite likely there's something wrong here...
According to the [Helix documentation](http://helix.apache.org/0.8.4-docs/Metrics.html), the MBean ZkClientMonitor has 4 histograms. Is there any way from inspecting the Heap Dump of finding out which 2 Histograms are consuming so much memory?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] PauloRelex commented on issue #918: 2 ZkClientPathMonitor
HistogramDynamicMetric using 100s of MBs each
Posted by GitBox <gi...@apache.org>.
PauloRelex commented on issue #918: 2 ZkClientPathMonitor HistogramDynamicMetric using 100s of MBs each
URL: https://github.com/apache/helix/issues/918#issuecomment-604982234
Eclipse Memory Analyser Tool, Linkerd:
Namerd:
^ that LinkedList has a ton of "Chunk" entries with pairs of long[].
The second long[] just has "20" in all indices, and the first long[] has values that appear to be timestamps _(not entirely sure they are timestamps, might just be a coincidence)_.
I see values that, being timestamps, are separated every few seconds and run at least until 9 months into the future:
1607901778746880 - 14 December 2020 1:22:58.746 AM GMT+02:00
1607901784900352 - 14 December 2020 1:23:04.900 AM GMT+02:00
1607901790380288 - 14 December 2020 1:23:10.380 AM GMT+02:00
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] PauloRelex edited a comment on issue #918: 2
ZkClientPathMonitor HistogramDynamicMetric using 100s of MBs each
Posted by GitBox <gi...@apache.org>.
PauloRelex edited a comment on issue #918: 2 ZkClientPathMonitor HistogramDynamicMetric using 100s of MBs each
URL: https://github.com/apache/helix/issues/918#issuecomment-604982234
Eclipse Memory Analyser Tool, Linkerd:
Namerd:
^ that LinkedList has a ton of "Chunk" entries with pairs of long[].
The second long[] just has "20" in all indices, and the first long[] has values that appear to be timestamps _(not entirely sure they are timestamps, might just be a coincidence)_.
I see values that, being timestamps, are separated precisely 6 seconds apart and run at least until 9 months into the future:
1607901778746880 - 14 December 2020 1:22:58.746 AM GMT+02:00
1607901784900352 - 14 December 2020 1:23:04.900 AM GMT+02:00
1607901790380288 - 14 December 2020 1:23:10.380 AM GMT+02:00
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org