[jira] [Updated] (CASSANDRA-4292) Per-disk I/O queues

["Yuki Morishita (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yuki Morishita updated CASSANDRA-4292:
--------------------------------------

    Attachment: 4292-v3.txt

Attaching v3.
Major changes:
- rename DiskWriter to IOManager and make it singleton.
- IOManager uses PriorityBlockingQueue in order to execute tasks based on priority. Memtable flushing task has higher priority than compaction task.

bq. need to use a single DiskWriter for both compaction and flushing or we lose on most of the benefits here. One solution: rename CompactionManager to IOManager, and use that. Another could be to move the DiskWriter into StorageService.

I'd rather keep CompactionManager as interface(API) to all compaction operations. So instead of renaming it to IOManager, I renamed DiskWriter to IOManager and use it from CompactionManager.

bq. compactionexecutor needs to be cleaned up since it's no longer serving the executor role. again, cleanup could be straightforward if we morph CM into IOManager (and merge CompactionExecutor + DiskWriter). Could be nice to get the kind of progress reporting on flushes that we now have on compaction.

Refactoring CompactionManager/CompactionExecutor is hard because of compaction lock. If it's possible, I'd like to do it later in separate ticket.

bq. DiskWriter: Can we use CopyOnWriteArrayList instead of synchronized block?

Done.
                
> Per-disk I/O queues
> -------------------
>
>                 Key: CASSANDRA-4292
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4292
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Yuki Morishita
>             Fix For: 1.2
>
>         Attachments: 4292-v2.txt, 4292-v3.txt, 4292.txt
>
>
> As noted in CASSANDRA-809, we have a certain amount of flush (and compaction) threads, which mix and match disk volumes indiscriminately.  It may be worth creating a tight thread -> disk affinity, to prevent unnecessary conflict at that level.
> OTOH as SSDs become more prevalent this becomes a non-issue.  Unclear how much pain this actually causes in practice in the meantime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira