You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Pavel Yaskevich (JIRA)" <ji...@apache.org> on 2011/03/24 19:13:06 UTC
[jira] [Updated] (CASSANDRA-1902) Migrate cached pages during
compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavel Yaskevich updated CASSANDRA-1902:
---------------------------------------
Attachment: CASSANDRA-1902-v4.patch
This patch makes lazy requests for the page cache info in the SSTableIdentityIterator.
BitSet is used in the BRAF to store pages that should be migrated (to avoid memory pressure).
Actual cache migration happens when SSTableReader gets created for the compacted file - if disk access mode is MMAP each of the segments will get advice using madvice, the same happens for "standard" mode but posix_fadvice is used.
Major change - when we have JNA and disk access mode is MMAP segments will be made using native mmap, this gives us a better control over the mapping (don't need to wait for GC to deallocate mapping, we can force it now using munmap method) this also means that mmap segment will be cleaned automatically so no more long hangs on GC and unused MappedByteBuffer won't hold the cache.
I was testing on the server with 2GB memory (hosted on rackspace): if data fits in the memory it will give you the same performance after compaction as it was before on the other side if data does not fit into memory I've experienced 3-4 times faster reads after compaction.
@tjake: can you test this on your Xen server?
> Migrate cached pages during compaction
> ---------------------------------------
>
> Key: CASSANDRA-1902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1902
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7.1
> Reporter: T Jake Luciani
> Assignee: T Jake Luciani
> Fix For: 0.8
>
> Attachments: 0001-CASSANDRA-1902-cache-migration-impl-with-config-option.txt, 1902-formatted.txt, 1902-per-column-migration-rebase2.txt, 1902-per-column-migration.txt, CASSANDRA-1902-v3.patch, CASSANDRA-1902-v4.patch
>
> Original Estimate: 32h
> Time Spent: 56h
> Remaining Estimate: 0h
>
> Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a pre-compacted CF during the compaction process. This is now important since CASSANDRA-1470 caches effectively nothing.
> For example an active CF being compacted hurts reads since nothing is cached in the new SSTable.
> The purpose of this ticket then is to make sure SOME data is cached from active CFs. This can be done my monitoring which Old SSTables are in the page cache and caching active rows in the New SStable.
> A simpler yet similar approach is described here: http://insights.oetiker.ch/linux/fadvise/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira