You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Craig Condit (JIRA)" <ji...@apache.org> on 2011/01/25 23:55:44 UTC

[jira] Created: (AMQ-3161) Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances

Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances
-------------------------------------------------------------------------------------------------------

                 Key: AMQ-3161
                 URL: https://issues.apache.org/jira/browse/AMQ-3161
             Project: ActiveMQ
          Issue Type: Bug
          Components: Message Store
    Affects Versions: 5.4.2
         Environment: CentOS 5.2
            Reporter: Craig Condit
            Priority: Critical


If the ActiveMQ Journal Checkpoint worker thread dies for any reason, any updates to KahaDB trigger the thread to be restarted. Unfortunately, there is a race condition between the Thread.isAlive() check and the restart of the thread, potentially leading to multiple instances of the thread being spawned.  We have observed in a busy environment as many as 8-10 instances of this thread, all waking up every 5 seconds to perform database checkpoints, bringing throughput on the broker to a halt.

The attached patch adds a lock around access to creation/destruction of this thread and prevents a second instance from being launched inadvertently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AMQ-3161) Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances

Posted by "Craig Condit (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AMQ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Craig Condit updated AMQ-3161:
------------------------------

    Attachment: checkpoint-thread.patch

This patch to MessageDatabase.java prevents accidental launching of multiple Journal checkpoint worker threads.

> Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances
> -------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3161
>                 URL: https://issues.apache.org/jira/browse/AMQ-3161
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.4.2
>         Environment: CentOS 5.2
>            Reporter: Craig Condit
>            Priority: Critical
>         Attachments: checkpoint-thread.patch
>
>
> If the ActiveMQ Journal Checkpoint worker thread dies for any reason, any updates to KahaDB trigger the thread to be restarted. Unfortunately, there is a race condition between the Thread.isAlive() check and the restart of the thread, potentially leading to multiple instances of the thread being spawned.  We have observed in a busy environment as many as 8-10 instances of this thread, all waking up every 5 seconds to perform database checkpoints, bringing throughput on the broker to a halt.
> The attached patch adds a lock around access to creation/destruction of this thread and prevents a second instance from being launched inadvertently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (AMQ-3161) Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances

Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AMQ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully resolved AMQ-3161.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 5.5.0

patch applied with thanks in r1063710

URL: http://svn.apache.org/viewvc?rev=1063710&view=rev

one minor mod, left unsynced check in store, such that locking overhead occurs only if thread needs a restart.

> Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances
> -------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3161
>                 URL: https://issues.apache.org/jira/browse/AMQ-3161
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.4.2
>         Environment: CentOS 5.2
>            Reporter: Craig Condit
>            Assignee: Gary Tully
>            Priority: Critical
>             Fix For: 5.5.0
>
>         Attachments: checkpoint-thread.patch
>
>
> If the ActiveMQ Journal Checkpoint worker thread dies for any reason, any updates to KahaDB trigger the thread to be restarted. Unfortunately, there is a race condition between the Thread.isAlive() check and the restart of the thread, potentially leading to multiple instances of the thread being spawned.  We have observed in a busy environment as many as 8-10 instances of this thread, all waking up every 5 seconds to perform database checkpoints, bringing throughput on the broker to a halt.
> The attached patch adds a lock around access to creation/destruction of this thread and prevents a second instance from being launched inadvertently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (AMQ-3161) Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances

Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AMQ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully reassigned AMQ-3161:
-------------------------------

    Assignee: Gary Tully

> Race condition in ActiveMQ Journal Checkpoint worker thread cleanup leads to multiple running instances
> -------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3161
>                 URL: https://issues.apache.org/jira/browse/AMQ-3161
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.4.2
>         Environment: CentOS 5.2
>            Reporter: Craig Condit
>            Assignee: Gary Tully
>            Priority: Critical
>         Attachments: checkpoint-thread.patch
>
>
> If the ActiveMQ Journal Checkpoint worker thread dies for any reason, any updates to KahaDB trigger the thread to be restarted. Unfortunately, there is a race condition between the Thread.isAlive() check and the restart of the thread, potentially leading to multiple instances of the thread being spawned.  We have observed in a busy environment as many as 8-10 instances of this thread, all waking up every 5 seconds to perform database checkpoints, bringing throughput on the broker to a halt.
> The attached patch adds a lock around access to creation/destruction of this thread and prevents a second instance from being launched inadvertently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.