Best practices for frequently updated columns

[Jeremy Jongsma <je...@barchart.com>]

We are building a historical timeseries database for stocks and futures,
with trade prices aggregated into daily bars (open, high, low, close values
for the day). The latest bar for each instrument needs to be updated as new
trades arrive on the realtime data feeds. Depending on the trading volume
for an instrument, some columns will be updated multiple times per second.

I've read comments about frequent column updates causing compaction issues
with Cassandra. What is the recommended Cassandra configuration / best
practices for usage scenarios like this?

Re: Best practices for frequently updated columns

[Philo Yang <ud...@gmail.com>]

The write performance of INSERT or UPDATE is very high in C*, but if you
update too often the row you update frequently will be in many SSTables so
the read latency and system load will be increased until these SSTables are
compacted into one single file.

I think you can use redis (or memcache) to cache the read request. Write
into C* and redis both and read only from redis for the latest bar. Read
from C* for previous bars if the memory size is bottleneck.

Thanks,
Philo Yang



2014-08-14 2:13 GMT+08:00 Robert Coli <rc...@eventbrite.com>:

> On Wed, Aug 13, 2014 at 8:01 AM, Jeremy Jongsma <je...@barchart.com>
> wrote:
>
>> I've read comments about frequent column updates causing compaction
>> issues with Cassandra. What is the recommended Cassandra configuration /
>> best practices for usage scenarios like this?
>>
>
> If your data is frequently UPDATEd, perhaps a log structured database with
> immutable data files is not for you?
>
> =Rob
>
>

Re: Best practices for frequently updated columns

[Robert Coli <rc...@eventbrite.com>]

On Wed, Aug 13, 2014 at 8:01 AM, Jeremy Jongsma <je...@barchart.com> wrote:

> I've read comments about frequent column updates causing compaction issues
> with Cassandra. What is the recommended Cassandra configuration / best
> practices for usage scenarios like this?
>

If your data is frequently UPDATEd, perhaps a log structured database with
immutable data files is not for you?

=Rob