You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefania (JIRA)" <ji...@apache.org> on 2015/09/01 04:28:48 UTC
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on
compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724615#comment-14724615 ]
Stefania commented on CASSANDRA-8630:
-------------------------------------
Thanks [~benedict] and [~JoshuaMcKenzie].
bq. Removed the readBytes method entirely
Are you sure? This method was introduced by CASSANDRA-1714, titled _zero-copy reads_. It copies BBs via {{FastByteOperations}} rather than reading into a buffer byte by byte. Bear in mind that {{ByteBufferUtil.read()}} is one of the most used read methods according to flight recorder.
bq. Removed the (as commented in TODO) unnecessary BufferedSegmentedFile.createReader
Thanks, I had totally forgot to remove this.
bq. MappedRegions.close now always calls channel.close, and returns any error thrown by it
Thanks for spotting it!
The remaining points are good. The new {{Throwables.perform}} API is really neat.
I've pushed a rebased version to Jenkins and I've asked on IRC to set-up Windows jobs for this branch. I've also launched a [cperf job|http://cstar.datastax.com/tests/id/35dbcc0e-5050-11e5-b3f6-42010af0688f]. I'll post the results once they are available.
> Faster sequential IO (on compaction, streaming, etc)
> ----------------------------------------------------
>
> Key: CASSANDRA-8630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8630
> Project: Cassandra
> Issue Type: Improvement
> Components: Core, Tools
> Reporter: Oleg Anastasyev
> Assignee: Stefania
> Labels: compaction, performance
> Fix For: 3.x
>
> Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, mmaped_uncomp_hotspot.png
>
>
> When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int).
> This is because default implementations of readShort,readLong, etc as well as their matching write* are implemented with numerous calls of byte by byte read and write.
> This makes a lot of syscalls as well.
> A quick microbench shows than just reimplementation of these methods in either way gives 8x speed increase.
> A patch attached implements RandomAccessReader.read<Type> and SequencialWriter.write<Type> methods in more efficient way.
> I also eliminated some extra byte copies in CompositeType.split and ColumnNameHelper.maxComponents, which were on my profiler's hotspot method list during tests.
> A stress tests on my laptop show that this patch makes compaction 25-30% faster on uncompressed sstables and 15% faster for compressed ones.
> A deployment to production shows much less CPU load for compaction.
> (I attached a cpu load graph from one of our production, orange is niced CPU load - i.e. compaction; yellow is user - i.e. not compaction related tasks)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)