You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Chris Goffinet (JIRA)" <ji...@apache.org> on 2010/12/06 21:09:10 UTC
[jira] Created: (CASSANDRA-1825) Separation of Data
(Cached/Non-Cached)
Separation of Data (Cached/Non-Cached)
--------------------------------------
Key: CASSANDRA-1825
URL: https://issues.apache.org/jira/browse/CASSANDRA-1825
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Fix For: 0.8
At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk.
Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from.
Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1825) Separation of Data
(Cached/Non-Cached)
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968395#action_12968395 ]
Chris Goffinet commented on CASSANDRA-1825:
-------------------------------------------
When I am talking about cache, I refer to page cache. If we had a stage that would be responsible for checking page cache, we would be forced to call mincore(), a system call on every request. This could get expensive very quick. It's worth prototyping to verify.
> Separation of Data (Cached/Non-Cached)
> --------------------------------------
>
> Key: CASSANDRA-1825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1825
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Chris Goffinet
> Fix For: 0.8
>
>
> At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk.
> Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from.
> Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Resolved] (CASSANDRA-1825) Separation of Data
(Cached/Non-Cached)
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-1825.
---------------------------------------
Resolution: Later
Fix Version/s: (was: 0.8)
> Separation of Data (Cached/Non-Cached)
> --------------------------------------
>
> Key: CASSANDRA-1825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1825
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Chris Goffinet
>
> At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk.
> Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from.
> Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1825) Separation of Data
(Cached/Non-Cached)
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968424#action_12968424 ]
Jonathan Ellis commented on CASSANDRA-1825:
-------------------------------------------
CASSANDRA-1379 is a similar issue but for row cache / BF check.
> Separation of Data (Cached/Non-Cached)
> --------------------------------------
>
> Key: CASSANDRA-1825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1825
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Chris Goffinet
> Fix For: 0.8
>
>
> At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk.
> Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from.
> Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1825) Separation of Data
(Cached/Non-Cached)
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968393#action_12968393 ]
Jonathan Ellis commented on CASSANDRA-1825:
-------------------------------------------
the easy solution that fits nicely w/ existing design is to just make the check-cache logic a separate stage (thread pool).
> Separation of Data (Cached/Non-Cached)
> --------------------------------------
>
> Key: CASSANDRA-1825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1825
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Chris Goffinet
> Fix For: 0.8
>
>
> At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk.
> Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from.
> Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.