You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Wayne Schroeder <ws...@pinsightmedia.com> on 2014/03/31 18:37:51 UTC
Row cache for writes
I found a lot of documentation about the read path for key and row caches, but I haven't found anything in regard to the write path. My app has the need to record a large quantity of very short lived temporal data that will expire within seconds and only have a small percentage of the rows accessed before they expire. Ideally, and I have done the math, I would like the data to never hit disk and just stay in memory once written until it expires. How might I accomplish this? I am not concerned about data consistency at all on this so if I could even avoid the commit log, that would be even better.
My main concern is that I don't see any evidence that writes end up in the cache—that it takes at least one read to get it into the cache. I also realize that, assuming I don't cause SSTable writes due to sheer quantity, that the data would be in memory anyway.
Has anyone done anything similar to this that could provide direction?
Wayne
Re: Row cache for writes
Posted by Tyler Hobbs <ty...@datastax.com>.
On Mon, Mar 31, 2014 at 11:37 AM, Wayne Schroeder <
wschroeder@pinsightmedia.com> wrote:
> I found a lot of documentation about the read path for key and row caches,
> but I haven't found anything in regard to the write path. My app has the
> need to record a large quantity of very short lived temporal data that will
> expire within seconds and only have a small percentage of the rows accessed
> before they expire. Ideally, and I have done the math, I would like the
> data to never hit disk and just stay in memory once written until it
> expires. How might I accomplish this?
>
It's not perfect, but set a short TTL on the data and set gc_grace_seconds
to 0 for the table. Tombstones will still be written to disk, but almost
everything will be discarded in its first compaction. You could also lower
the min compaction threshold for size-tiered compaction to 2 to force
compactions to happen more quickly.
> I am not concerned about data consistency at all on this so if I could
> even avoid the commit log, that would be even better.
>
You can set durable_writes = false for the keyspace.
>
> My main concern is that I don't see any evidence that writes end up in the
> cache--that it takes at least one read to get it into the cache. I also
> realize that, assuming I don't cause SSTable writes due to sheer quantity,
> that the data would be in memory anyway.
>
> Has anyone done anything similar to this that could provide direction?
>
Writes invalidate row cache entries, so that's not what you want.
--
Tyler Hobbs
DataStax <http://datastax.com/>
Re: Row cache for writes
Posted by Ashok Ghosh <as...@gmail.com>.
On Mar 31, 2014 12:38 PM, "Wayne Schroeder" <ws...@pinsightmedia.com>
wrote:
> I found a lot of documentation about the read path for key and row caches,
> but I haven't found anything in regard to the write path. My app has the
> need to record a large quantity of very short lived temporal data that will
> expire within seconds and only have a small percentage of the rows accessed
> before they expire. Ideally, and I have done the math, I would like the
> data to never hit disk and just stay in memory once written until it
> expires. How might I accomplish this? I am not concerned about data
> consistency at all on this so if I could even avoid the commit log, that
> would be even better.
>
> My main concern is that I don't see any evidence that writes end up in the
> cache--that it takes at least one read to get it into the cache. I also
> realize that, assuming I don't cause SSTable writes due to sheer quantity,
> that the data would be in memory anyway.
>
> Has anyone done anything similar to this that could provide direction?
>
> Wayne
>
>
Re: Row cache for writes
Posted by Wayne Schroeder <ws...@pinsightmedia.com>.
Perhaps I should clarify my question. Is this possible / how might I accomplish this with cassandra?
Wayne
On Mar 31, 2014, at 12:58 PM, Robert Coli <rc...@eventbrite.com>>
wrote:
On Mon, Mar 31, 2014 at 9:37 AM, Wayne Schroeder <ws...@pinsightmedia.com>> wrote:
I found a lot of documentation about the read path for key and row caches, but I haven't found anything in regard to the write path. My app has the need to record a large quantity of very short lived temporal data that will expire within seconds and only have a small percentage of the rows accessed before they expire. Ideally, and I have done the math, I would like the data to never hit disk and just stay in memory once written until it expires. How might I accomplish this?
http://en.wikipedia.org/wiki/Memcached
=Rob
Re: Row cache for writes
Posted by Robert Coli <rc...@eventbrite.com>.
On Mon, Mar 31, 2014 at 9:37 AM, Wayne Schroeder <
wschroeder@pinsightmedia.com> wrote:
> I found a lot of documentation about the read path for key and row caches,
> but I haven't found anything in regard to the write path. My app has the
> need to record a large quantity of very short lived temporal data that will
> expire within seconds and only have a small percentage of the rows accessed
> before they expire. Ideally, and I have done the math, I would like the
> data to never hit disk and just stay in memory once written until it
> expires. How might I accomplish this?
http://en.wikipedia.org/wiki/Memcached
=Rob