[jira] [Commented] (CASSANDRA-11940) Look into better default file_cache_size for 2.2

["Eric Evans (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331973#comment-15331973 ] 

Eric Evans commented on CASSANDRA-11940:
----------------------------------------

In the plot below, nodes cerium, praseodymium (worst. hostname. evar.), and xenon are under a workload consisting primarily of random reads.  At ~11:00, {{disk_access_mode}} is changed from {{auto}} to {{mmap_index_only}} on cerium and praseodymium only.  When this happens, disk read throughput drops dramatically, as does the number of major page faults generated by Cassandra (from ~280/s to ~15/s).

Over the course of this run, I increased {{file_cache_size_in_mb}} from 512, to 768, 1024, and 2048, with no observable difference to disk read throughput, or the rate of major page faults.

These machines are 12-way (w/ hyperthreading), 16G RAM, and rotational disks.  They have 4G heaps (though raised to 6G for the {{file_cache_size_in_mb: 2048}} test)

!abnormal disk read throughput.png|width=960!

{noformat}
CREATE TABLE "local_group_wikipedia_T_parsoid_html".data (
    "_domain" text,
    key text,
    rev int,
    tid timeuuid,
    "_del" timeuuid,
    "content-location" text,
    "content-sha256" blob,
    "content-type" text,
    "latestTid" timeuuid,
    tags set<text>,
    value blob,
    PRIMARY KEY (("_domain", key), rev, tid)
) WITH CLUSTERING ORDER BY (rev DESC, tid DESC)
    AND bloom_filter_fp_chance = 0.1
    AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
    AND comment = ''
    AND compaction = {'tombstone_threshold': '0.02', 'unchecked_tombstone_compaction': 'true', 'base_time_seconds': '45', 'class': 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
    AND compression = {'chunk_length_kb': '512', 'sstable_compression': 'org.apache.cassandra.io.compress.DeflateCompressor'}
    AND dclocal_read_repair_chance = 0.0
    AND default_time_to_live = 0
    AND gc_grace_seconds = 86400
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';
{noformat}

> Look into better default file_cache_size for 2.2
> ------------------------------------------------
>
>                 Key: CASSANDRA-11940
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11940
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: T Jake Luciani
>            Assignee: T Jake Luciani
>             Fix For: 2.2.x
>
>         Attachments: abnormal disk read throughput.png
>
>
> CASSANDRA-8464 added support for mmapped decompression where in version <= 2.1 the reads were all decompressed in standard heap buffers.
> Since the usage of the file_cache is based solely on the buffer capacity we should/can make this much larger in 2.2 when the disk access mode is mmap.  
> The downside of this cache being too small is made worse by 8464 since the buffers are mmapped/unmapped causing explicit page faults.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)