[jira] [Commented] (CASSANDRA-8709) Convert SequentialWriter from using RandomAccessFile to nio channel

["Joshua McKenzie (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345249#comment-14345249 ] 

Joshua McKenzie commented on CASSANDRA-8709:
--------------------------------------------

Branch updated.

bq. CompressedSW.flushData() calls crcMetadata.append(compressed.buffer.array() ... is clearer.
Fixed.  Left the rewind since appendDirect relies on .position()

bq. In DataIntegrityMetadata, your new appendDirect call shouldn't be using mark and reset since it's racy. Better to .duplicate() the input buffer.
Switched to duplicated ByteBuffer and mark/reset on that as the counters should be of local use only and thus no threat from a raciness perspective.

bq. In LZ4Compressor.compress() the source length should be using .remaining() not .limit()
Good catch - fixed.

bq. All of your non-direct byte buffer code makes me nervous since you are accessing .array()...
I went ahead and swapped all of those calls to the appendDirect form.

I also uncommented a block in CompressorTest that snuck into the patch file.

bq. Write test for CompressedSW across all compressors
Added. The unit tests uncovered what appears to be a bug in CompressedSequentialWriter.resetAndTruncate with resetting to a mark that's at buffer-aligned length. I backported that test into current 2.0/2.1 and the same error occurs; we don't mark the current buffered data as dirty on resetAndTruncate so if we reset to the chunkOffset with a full buffer it's never marked dirty from a subsequent write and reBuffer just drops the data.  I'll open a ticket for 2.0.13 to get that fix in once we've confirmed it here.

> Convert SequentialWriter from using RandomAccessFile to nio channel
> -------------------------------------------------------------------
>
>                 Key: CASSANDRA-8709
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8709
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Joshua McKenzie
>            Assignee: Joshua McKenzie
>              Labels: Windows
>             Fix For: 3.0
>
>
> For non-mmap'ed I/O on Windows, using nio channels will give us substantially more flexibility w/regards to renaming and moving files around while writing them.  This change in conjunction with CASSANDRA-4050 should allow us to remove the Windows bypass code in SSTableRewriter for non-memory-mapped I/O.
> In general, migrating from instances of RandomAccessFile to nio channels will help make Windows and linux behavior more consistent.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)