You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Zephyr Guo (Jira)" <ji...@apache.org> on 2019/08/30 13:45:00 UTC
[jira] [Updated] (CASSANDRA-15295) Running into deadlock when do
CommitLog initialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zephyr Guo updated CASSANDRA-15295:
-----------------------------------
Attachment: jstack.log
> Running into deadlock when do CommitLog initialization
> ------------------------------------------------------
>
> Key: CASSANDRA-15295
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15295
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Priority: Normal
> Attachments: image-2019-08-30-21-25-17-638.png, image-2019-08-30-21-29-48-623.png, image-2019-08-30-21-30-11-683.png, image-2019-08-30-21-40-43-748.png, jstack.log
>
>
> Recently, I found a cassandra(3.11.4) node stuck in STARTING status for a long time.
> I used jstack to saw what happened. The main thread stuck in *AbstractCommitLogSegmentManager.awaitAvailableSegment*
> !image-2019-08-30-21-40-43-748.png|thumbnail!
> The strange thing is COMMIT-LOG-ALLOCATOR thread state was runnable but it was not actually running.
> !image-2019-08-30-21-30-11-683.png|thumbnail!
> And then I used pstack to troubleshoot. I found COMMIT-LOG-ALLOCATOR block on java class initialization.
> !image-2019-08-30-21-25-17-638.png|thumbnail!
> This is a deadlock obviously. CommitLog waits for a CommitLogSegment when initializing. In this moment, the CommitLog class is not initialized and the main thread holds the class lock. After that, COMMIT-LOG-ALLOCATOR creates a CommitLogSegment with exception and call *CommitLog.handleCommitError*(static method). COMMIT-LOG-ALLOCATOR will stick on this line because CommitLog class is still initializing.
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org