You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Hannu Kröger <hk...@gmail.com> on 2017/12/19 11:32:33 UTC

Nodetool compactionstats hangs

Hi,

I opened a ticket about nodetool compactionstats hanging:
https://issues.apache.org/jira/browse/CASSANDRA-14130

The root cause seems to be JMX metric fetching hanging. I was able to
replicate it on this problematic like this:

Welcome to JMX terminal. Type "help" for available commands.
$>open localhost:7199
#Connection to localhost:7199 is opened
$>bean org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
#bean is set to
org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
$>get Value
#mbean = org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName:

// This command never finishes.


This happens only on one node AFAIK. What could possibly be the problem?

Hannu

Re: Nodetool compactionstats hangs

Posted by Hannu Kröger <hk...@gmail.com>.
Hi,

Sure! I attached the jstack dumps on the ticket.

Hannu

On 19 December 2017 at 14:38:45, Jeff Jirsa (jjirsa@gmail.com) wrote:

Can you grab a thread dump with jstack as well?

--
Jeff Jirsa


On Dec 19, 2017, at 3:32 AM, Hannu Kröger <hk...@gmail.com> wrote:

Hi,

I opened a ticket about nodetool compactionstats hanging:
https://issues.apache.org/jira/browse/CASSANDRA-14130

The root cause seems to be JMX metric fetching hanging. I was able to
replicate it on this problematic like this:

Welcome to JMX terminal. Type "help" for available commands.
$>open localhost:7199
#Connection to localhost:7199 is opened
$>bean org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
#bean is set to
org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
$>get Value
#mbean = org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName:

// This command never finishes.


This happens only on one node AFAIK. What could possibly be the problem?

Hannu

Re: Nodetool compactionstats hangs

Posted by Jeff Jirsa <jj...@gmail.com>.
Can you grab a thread dump with jstack as well? 

-- 
Jeff Jirsa


> On Dec 19, 2017, at 3:32 AM, Hannu Kröger <hk...@gmail.com> wrote:
> 
> Hi,
> 
> I opened a ticket about nodetool compactionstats hanging:
> https://issues.apache.org/jira/browse/CASSANDRA-14130
> 
> The root cause seems to be JMX metric fetching hanging. I was able to replicate it on this problematic like this:
> Welcome to JMX terminal. Type "help" for available commands.
> $>open localhost:7199
> #Connection to localhost:7199 is opened
> $>bean org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
> #bean is set to org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName
> $>get Value
> #mbean = org.apache.cassandra.metrics:type=Compaction,name=PendingTasksByTableName:
> // This command never finishes.
> 
> This happens only on one node AFAIK. What could possibly be the problem?
> Hannu