You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2011/03/15 19:11:38 UTC

[jira] Created: (CASSANDRA-2333) Clean up thread pool and queue sizes

Clean up thread pool and queue sizes
------------------------------------

                 Key: CASSANDRA-2333
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
             Project: Cassandra
          Issue Type: Bug
            Reporter: Jonathan Ellis
            Priority: Minor


Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting with Core threads, adding threads up to Max as tasks arrive, then queuing any additional.  This is not correct:

{noformat}
    If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
    If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
    If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.
{noformat}

CASSANDRA-2178 fixed this in one place but made it worse by default since most people run with a single data dir, meaning as soon as you have multiple CFs flushing (or a single one with indexes) then you will start blocking writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2333) Clean up thread pool and queue sizes

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-2333:
--------------------------------------

    Attachment: 2333.txt

Patch to remove use of maximumpoolsize in favor of core size + timing out core threads.  Also adds memtable_flush_queue_size configuration directive, defaulting to 4.

> Clean up thread pool and queue sizes
> ------------------------------------
>
>                 Key: CASSANDRA-2333
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jonathan Ellis
>            Priority: Minor
>             Fix For: 0.7.5
>
>         Attachments: 2333.txt
>
>
> Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting with Core threads, adding threads up to Max as tasks arrive, then queuing any additional.  This is not correct:
> {noformat}
>     If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
>     If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
>     If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.
> {noformat}
> CASSANDRA-2178 fixed this in one place but made it worse by default since most people run with a single data dir, meaning as soon as you have multiple CFs flushing (or a single one with indexes) then you will start blocking writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2333) Clean up thread pool and queue sizes

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007102#comment-13007102 ] 

Brandon Williams commented on CASSANDRA-2333:
---------------------------------------------

+1, increased flush queue size is enough to prevent blocking writes at all with stress and an index.

> Clean up thread pool and queue sizes
> ------------------------------------
>
>                 Key: CASSANDRA-2333
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>            Priority: Minor
>             Fix For: 0.7.5
>
>         Attachments: 2333.txt
>
>
> Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting with Core threads, adding threads up to Max as tasks arrive, then queuing any additional.  This is not correct:
> {noformat}
>     If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
>     If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
>     If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.
> {noformat}
> CASSANDRA-2178 fixed this in one place but made it worse by default since most people run with a single data dir, meaning as soon as you have multiple CFs flushing (or a single one with indexes) then you will start blocking writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2333) Clean up thread pool and queue sizes

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007160#comment-13007160 ] 

Hudson commented on CASSANDRA-2333:
-----------------------------------

Integrated in Cassandra-0.7 #386 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])
    

> Clean up thread pool and queue sizes
> ------------------------------------
>
>                 Key: CASSANDRA-2333
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>            Priority: Minor
>             Fix For: 0.7.5
>
>         Attachments: 2333.txt
>
>
> Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting with Core threads, adding threads up to Max as tasks arrive, then queuing any additional.  This is not correct:
> {noformat}
>     If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
>     If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
>     If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.
> {noformat}
> CASSANDRA-2178 fixed this in one place but made it worse by default since most people run with a single data dir, meaning as soon as you have multiple CFs flushing (or a single one with indexes) then you will start blocking writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira