[jira] [Comment Edited] (CASSANDRA-9403) Experiment with skipping file syncs during unit tests to reduce test time

["Stefania (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551589#comment-14551589 ] 

Stefania edited comment on CASSANDRA-9403 at 5/20/15 1:09 AM:
--------------------------------------------------------------

Regarding this utest failure, http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/, I would simply store {{s.bytesOnDisk()}} in a variable just like {{startStorageMetricsLoad}}.

As far as I understand the code, once the sstable is compacted and replaced, at line 285, the reference is released and there is nothing stopping the deleting task from removing the files, see DescriptorTypeTidy in SSTableReader and SSTableDeletingTask. SSTableDeletingTask runs asynchronously.

I can reliably fail that assertion by calling SSTableDeletingTask.waitForDeletions(); just before it.

Alternatively you can try increasing the reference count for s, {{s.selfRef().tryRef()}}, and this should keep the files but you also need to release it at the end of the test, {{s.selfRef().release()}}.



was (Author: stefania):
Regarding this utest failure, http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/, I would simply store {{s.bytesOnDisk()}} in a variable just like {{startStorageMetricsLoad}}.

As far as I understand the code, once the sstable is compacted and replaced, at line 285, the reference is released and there is nothing stopping the deleting task from removing the files, see DescriptorTypeTidy in SSTableReader and SSTableDeletingTask. The SSTableDeletingTask runs asynchronously.

I can reliably fail that assertion by calling SSTableDeletingTask.waitForDeletions(); just before it.

Alternatively you can try increasing the reference count for s, {{s.selfRef().tryRef()}}, and this should keep the files but you also need to release it at the end of the test, {{s.selfRef().release()}}.


> Experiment with skipping file syncs during unit tests to reduce test time
> -------------------------------------------------------------------------
>
>                 Key: CASSANDRA-9403
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9403
>             Project: Cassandra
>          Issue Type: Test
>            Reporter: Ariel Weisberg
>            Assignee: Ariel Weisberg
>
> Some environments have ridiculous outliers for disk syncing. 20 seconds ridiculous.
> Unit tests aren't testing crash safety so it is a pointless exercise.
> Instead we could intercept calls to sync files and check whether it looks like the sync would succeed. Check that the things are not null, mapped, closed etc. Outside of units tests it can go straight to the regular sync call.
> I would also like to have the disks for unit and dtests mounted with barrier=0,noatime,nodiratime to further reduce susceptibility to outliers. We aren't going to recover these nodes if they crash/restart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)