You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Created) (JIRA)" <ji...@apache.org> on 2011/11/12 01:24:51 UTC
[jira] [Created] (ACCUMULO-140) Prevent linux from caching write
ahead log files
Prevent linux from caching write ahead log files
------------------------------------------------
Key: ACCUMULO-140
URL: https://issues.apache.org/jira/browse/ACCUMULO-140
Project: Accumulo
Issue Type: Improvement
Components: logger
Reporter: Keith Turner
Assignee: Eric Newton
While at ApacheCon I attended a talk about Cassandra. The developer mentioned using posix_fadvise to prevent linux from storing edit/write ahead log data in the page cache. This sounds great. Accumulo can do this for its write ahead logs.
Using posix_fadvise was also mentioned for compactions, this is something Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-140) Prevent linux from caching write
ahead log files
Posted by "Keith Turner (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated ACCUMULO-140:
----------------------------------
Fix Version/s: 1.5.0
> Prevent linux from caching write ahead log files
> ------------------------------------------------
>
> Key: ACCUMULO-140
> URL: https://issues.apache.org/jira/browse/ACCUMULO-140
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Keith Turner
> Assignee: Eric Newton
> Fix For: 1.5.0
>
>
> While at ApacheCon I attended a talk about Cassandra. The developer mentioned using posix_fadvise to prevent linux from storing edit/write ahead log data in the page cache. This sounds great. Accumulo can do this for its write ahead logs.
> Using posix_fadvise was also mentioned for compactions, this is something Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-140) Prevent linux from caching write
ahead log files
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148891#comment-13148891 ]
Keith Turner commented on ACCUMULO-140:
---------------------------------------
Would it be worthwhile adding fadvise like support to HDFS, where it can be given hints about the application plans to access a file?
> Prevent linux from caching write ahead log files
> ------------------------------------------------
>
> Key: ACCUMULO-140
> URL: https://issues.apache.org/jira/browse/ACCUMULO-140
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Keith Turner
> Assignee: Eric Newton
>
> While at ApacheCon I attended a talk about Cassandra. The developer mentioned using posix_fadvise to prevent linux from storing edit/write ahead log data in the page cache. This sounds great. Accumulo can do this for its write ahead logs.
> Using posix_fadvise was also mentioned for compactions, this is something Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-140) Prevent linux from caching write
ahead log files
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148915#comment-13148915 ]
Todd Lipcon commented on ACCUMULO-140:
--------------------------------------
The current HDFS support (HDFS-2465) doesn't expose hinting as a client API -- it's a DN-wide setting. The plan is to later add some kind of hint flag to the protocol so you could push down the hint from the client.
> Prevent linux from caching write ahead log files
> ------------------------------------------------
>
> Key: ACCUMULO-140
> URL: https://issues.apache.org/jira/browse/ACCUMULO-140
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Keith Turner
> Assignee: Eric Newton
>
> While at ApacheCon I attended a talk about Cassandra. The developer mentioned using posix_fadvise to prevent linux from storing edit/write ahead log data in the page cache. This sounds great. Accumulo can do this for its write ahead logs.
> Using posix_fadvise was also mentioned for compactions, this is something Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-140) Prevent linux from caching write
ahead log files
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148894#comment-13148894 ]
Keith Turner commented on ACCUMULO-140:
---------------------------------------
See HDFS-2465 and HADOOP-7714
> Prevent linux from caching write ahead log files
> ------------------------------------------------
>
> Key: ACCUMULO-140
> URL: https://issues.apache.org/jira/browse/ACCUMULO-140
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Keith Turner
> Assignee: Eric Newton
>
> While at ApacheCon I attended a talk about Cassandra. The developer mentioned using posix_fadvise to prevent linux from storing edit/write ahead log data in the page cache. This sounds great. Accumulo can do this for its write ahead logs.
> Using posix_fadvise was also mentioned for compactions, this is something Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira