You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Peter Schuller (Created) (JIRA)" <ji...@apache.org> on 2012/02/23 04:21:55 UTC
[jira] [Created] (CASSANDRA-3950) support trickling fsync() on
writes
support trickling fsync() on writes
-----------------------------------
Key: CASSANDRA-3950
URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Peter Schuller
Assignee: Peter Schuller
Fix For: 1.1.0
Attachments: CASSANDRA-3950-1.1.txt
Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216592#comment-13216592 ]
Peter Schuller commented on CASSANDRA-3950:
-------------------------------------------
Apologies again. Obviously I should not be submitting patches when in the middle of other things. I will fix that (which is trivial) and submit the new patch when properly tested and looked over.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3950:
--------------------------------------
Reviewer: xedin
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219026#comment-13219026 ]
Peter Schuller commented on CASSANDRA-3950:
-------------------------------------------
And of course the value should have been lower anyway. :/
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1-v3.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Schuller updated CASSANDRA-3950:
--------------------------------------
Attachment: CASSANDRA-3950-1.1-v3.txt
{{v3}} attached. Third time's the charm. I made it {{in_kb}} instead too, because I realized that megabyte resolution is not necessarily enough for really low-latency cases on a modern SSD (1 MB for a 90 MB/sec seq. writing SSD is 11 milliseconds).
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1-v3.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215041#comment-13215041 ]
Pavel Yaskevich commented on CASSANDRA-3950:
--------------------------------------------
few last nits:
# inside of your if condition you still use bytesSinceCacheFlush instead of bytesSinceTrickleFsync to set it to 0.
# "Requires JNA." can be removed from comment in conf/cassandra.yaml since build-in getFD().sync() is used.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219012#comment-13219012 ]
Pavel Yaskevich commented on CASSANDRA-3950:
--------------------------------------------
+1 with final change required: default 1000000 should be changed to 10240 (which is 10 MB value in kilobytes, it will be multiplied by 1024 in the SW constructor which would give 10MB value in bytes = 10485760).
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1-v3.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216594#comment-13216594 ]
Pavel Yaskevich commented on CASSANDRA-3950:
--------------------------------------------
No problem, Peter. Take your time.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219028#comment-13219028 ]
Pavel Yaskevich commented on CASSANDRA-3950:
--------------------------------------------
yeah, this is exactly two of my points.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1-v3.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219025#comment-13219025 ]
Peter Schuller commented on CASSANDRA-3950:
-------------------------------------------
I'm fine with that. I intentionally kept it even for simplicity, but on second though it's actually a good idea to keep it nicely aligned (aligning with SSD page size is good, and an even MB is very likely to be).
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1-v3.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214630#comment-13214630 ]
Pavel Yaskevich commented on CASSANDRA-3950:
--------------------------------------------
The problems with your patch:
# you use bytesSinceCacheFlush instead of bytesSinceTrickleFsync in flushInternal() method.
# trickleFsyncByteInterval should be bumped by 1024 * 1024 in the SW constructor to make it bytes because bytesSinceTrickleFsync is counted in bytes granularity.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Schuller updated CASSANDRA-3950:
--------------------------------------
Attachment: CASSANDRA-3950-1.1.txt
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3950) support trickling fsync() on
writes
Posted by "Peter Schuller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Schuller updated CASSANDRA-3950:
--------------------------------------
Attachment: CASSANDRA-3950-1.1-v2.txt
Apologies. Attaching fixed version.
> support trickling fsync() on writes
> -----------------------------------
>
> Key: CASSANDRA-3950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3950
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Fix For: 1.1.0
>
> Attachments: CASSANDRA-3950-1.1-v2.txt, CASSANDRA-3950-1.1.txt
>
>
> Attaching a patch to support fsync():ing every N megabytes of data written using sequential writers. The motivation is to avoid the kernel flushing out pages in bulk.
> It makes sense for both platters and SSD:s, but it's particularly good for SSD:s because the negative consequences of fsync():ing more often are much more limited than with platters, and the *need* is to some extent greater because of the fact that with SSD:s you're much more likely to be e.g. streaming data quickly or compacting quickly, since you're not having to throttle everything as extremely as with platters, and you easily write fast enough for this to be a problem if you're targetting good latency at the outliers.
> I'm nominating it for 1.1.0 because, if disabled, the probability of this being a regression seems very low.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira