[jira] [Comment Edited] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin lock that allocates memory

["graham sanderson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061521#comment-14061521 ] 

graham sanderson edited comment on CASSANDRA-7546 at 7/15/14 1:29 AM:
----------------------------------------------------------------------

Prior email thread comment from [~benedict]

{quote}
"I favour an asynchronous approach permitting only one modifying thread access to the
structure at a time, with each competing modification simply chaining their
to-be-merged state to a pending-list, which is repaired at read time if it
isn't merged before then. i.e. each writer attempts to take an exclusive
lock on modification, and if it succeeds it merges its own state and any
other pending state it sees on entry; if it fails to take the lock it
appends the unmerged modification to a simple list, and returns success.

I plan to address this, but probably not until after 3.0, so if you're in a
rush it may be worth exploring one of these alternative approaches. The
simplest, least modifying solution, is probably to use unsafe to acquire
the object monitor if we fail the first cas (but continue to execute the
same codepath), so that we simply gate the amount of contention we can
experience without incurring any extra costs in the common case."
{quote}

Note that I haven't tested my suggestion on 2.1 yet but I will do so


was (Author: graham sanderson):
Prior email thread comment from [~benedict]

"I favour an asynchronous approach permitting only one modifying thread access to the
structure at a time, with each competing modification simply chaining their
to-be-merged state to a pending-list, which is repaired at read time if it
isn't merged before then. i.e. each writer attempts to take an exclusive
lock on modification, and if it succeeds it merges its own state and any
other pending state it sees on entry; if it fails to take the lock it
appends the unmerged modification to a simple list, and returns success.

I plan to address this, but probably not until after 3.0, so if you're in a
rush it may be worth exploring one of these alternative approaches. The
simplest, least modifying solution, is probably to use unsafe to acquire
the object monitor if we fail the first cas (but continue to execute the
same codepath), so that we simply gate the amount of contention we can
experience without incurring any extra costs in the common case."

Note that I haven't tested my suggestion on 2.1 yet but I will do so

> AtomicSortedColumns.addAllWithSizeDelta has a spin lock that allocates memory
> -----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7546
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: graham sanderson
>         Attachments: suggestion1.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update, then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some fairly staggering memory growth (the more cores on your machine the worst it gets).
> Whilst many usage patterns don't do highly concurrent updates to the same partition, hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory allocation rates can be seen (especially when the updates being hinted are small updates to different partitions which can happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)