You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Benedict (JIRA)" <ji...@apache.org> on 2018/11/27 11:53:00 UTC
[jira] [Comment Edited] (CASSANDRA-14748) Recycler$WeakOrderQueue
occupies Heap
[ https://issues.apache.org/jira/browse/CASSANDRA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613112#comment-16613112 ]
Benedict edited comment on CASSANDRA-14748 at 11/27/18 11:52 AM:
-----------------------------------------------------------------
I guess I should go and revisit the WeakOrderQueue and bound its laziness, as this is a recurring them - on the Netty project bug tracker as well.
It was originally assumed that all recycling would happen on the same thread, but that we must support occasional recycles from another thread. So it was anticipated that the WeakOrderQueue would be used infrequently. But, it seems to get exercised often in some use cases - ours apparently being one. It's actually probably not too tricky to resolve this, so I'll see about opening a ticket on the netty project.
We should anyway try to track down where we are doing this in the project, as it is suboptimal, and not how the recycler is intended to be used.
Could you post the full heap dump, or at least the full class histogram? The WeakOrderQueue should be holding on to some child objects.
was (Author: benedict):
I guess I should go and revisit the WeakOrderQueue and bound its laziness, as this is a recurring them - on the Netty project bug tracker as well.
When I wrote it, the spec I was given had that it was intended for all recycling to happen on the same thread, but that we must support occasional recycles from another thread. So it was anticipated that the WeakOrderQueue would be used infrequently. But, it seems to get exercised often in some use cases - ours apparently being one. It's actually probably not too tricky to resolve this, so I'll see about opening a ticket on the netty project.
We should anyway try to track down where we are doing this in the project, as it is suboptimal, and not how the recycler is intended to be used.
Could you post the full heap dump, or at least the full class histogram? The WeakOrderQueue should be holding on to some child objects.
> Recycler$WeakOrderQueue occupies Heap
> -------------------------------------
>
> Key: CASSANDRA-14748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14748
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: The netty Cassandra using is netty-all-4.0.39.Final.jar
> Reporter: HX
> Priority: Major
> Fix For: 3.10
>
>
> Heap constantly high on some of the nodes in the cluster, I dump the heap and open it through Eclipse Memory Analyzer, looks like Recycler$WeakOrderQueue occupies most of the heap.
>
> ||Package||Retained Heap||Retained Heap, %||# Top Dominators||
> |!/jira/icons/i5.gif! <all>|7,078,140,136|100.00%|379,627|
> |io|5,665,035,800|80.04%|13,306|
> |netty|5,665,035,800|80.04%|13,306|
> |util|5,568,107,344|78.67%|2,965|
> |Recycler$WeakOrderQueue|4,950,021,544|69.93%|2,169|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org