You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Chris Goffinet (JIRA)" <ji...@apache.org> on 2009/09/01 06:09:32 UTC
[jira] Created: (CASSANDRA-404) SSTable write lock not releasing
SSTable write lock not releasing
--------------------------------
Key: CASSANDRA-404
URL: https://issues.apache.org/jira/browse/CASSANDRA-404
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.4
Reporter: Chris Goffinet
Attachments: stack.log.gz
Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred. We also are seeing a race condition with ConcurrentLinkedHashMap using appendToTail.
We are currently investigating the ssTable lock. We could remove the ConcurrentLinkedHashMap for now until that's resolved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-404) SSTable write lock not releasing
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Goffinet updated CASSANDRA-404:
-------------------------------------
Attachment: stack.log.gz
Our stack trace from the running node in question.
> SSTable write lock not releasing
> --------------------------------
>
> Key: CASSANDRA-404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-404
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Reporter: Chris Goffinet
> Attachments: stack.log.gz
>
>
> Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred. We also are seeing a race condition with ConcurrentLinkedHashMap using appendToTail.
> We are currently investigating the ssTable lock. We could remove the ConcurrentLinkedHashMap for now until that's resolved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-404) SSTable write lock not releasing
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Goffinet resolved CASSANDRA-404.
--------------------------------------
Resolution: Invalid
We are using a fair readwrite lock, this is why it's hanging. See CASSANDRA-405
> SSTable write lock not releasing
> --------------------------------
>
> Key: CASSANDRA-404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-404
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Reporter: Chris Goffinet
> Attachments: stack.log.gz
>
>
> Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred.
> We are currently investigating the ssTable lock.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-404) SSTable write lock not releasing
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Goffinet updated CASSANDRA-404:
-------------------------------------
Description:
Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred.
We are currently investigating the ssTable lock.
was:
Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred. We also are seeing a race condition with ConcurrentLinkedHashMap using appendToTail.
We are currently investigating the ssTable lock. We could remove the ConcurrentLinkedHashMap for now until that's resolved.
> SSTable write lock not releasing
> --------------------------------
>
> Key: CASSANDRA-404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-404
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Reporter: Chris Goffinet
> Attachments: stack.log.gz
>
>
> Today we saw a node lock up when trying to serve reads. Current investigation shows that some threads are trying to acquire a ssTable_.lock(). This means that somehow Cassandra has gotten into a state with a write lock that has not been released (blocking read locks). Thrift connections started piling up after this occurred.
> We are currently investigating the ssTable lock.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.