You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Lapo Luchini <la...@lapo.it> on 2022/12/29 08:55:06 UTC
Best compaction strategy for rarely used data
Hi, I have a table which gets (a lot of) data that is written once and
very rarely read (it is used for data that is mandatory for regulatory
reasons), and almost never deleted.
I'm using the default SCTS as at the time I didn't know any better, but
SSTables size are getting huge, which is a problem because they both are
getting to the size of the available disk and both because I'm using a
snapshot-based system to backup the node (and thus compacting a huge
SSTable into an even bigger one generates a lot of traffic for
mostly-old data).
I'm thinking about switching to LCS (mainly to solve the size issue),
but I read that it is "optimized for read heavy workloads […] not a good
choice for immutable time series data". Given that I don't really care
about write nor read speed, but would like SSTables size to have a upper
limit, would this strategy still be the best?
PS: Googling around a strategy called "incremental compaction" (ICS)
keeps getting in results, but that's only available in ScyllaDB, right?
--
Lapo Luchini
lapo@lapo.it
RE: Best compaction strategy for rarely used data
Posted by onmstester onmstester via user <us...@cassandra.apache.org>.
Another solution: distribute data in more tables, for example you could create multiple tables based on value or hash_bucket of one of the columns, by doing this current data volume and compaction overhead would be divided to the number of underlying tables. Although there is a limitation for number of tables in Cassandra (a few hundreds).
I wish STCS simply had a limitation for maximum sstable size so sstables bigger that this limit would not be compacted at all, that would have solved most of similar problems?!
Sent using https://www.zoho.com/mail/
---- On Fri, 30 Dec 2022 21:43:27 +0330 Durity, Sean R via user <us...@cassandra.apache.org> wrote ---
Yes, clean-up will reduce the disk space on the existing nodes by re-writing only the data that the node now owns into new sstables.
Sean R. Durity
DB Solutions
Staff Systems Engineer – Cassandra
From: Lapo Luchini <ma...@lapo.it>
Sent: Friday, December 30, 2022 4:12 AM
To: mailto:user@cassandra.apache.org
Subject: [EXTERNAL] Re: Best compaction strategy for rarely used data
On 2022-12-29 21: 54, Durity, Sean R via user wrote: > At some point you will end up with large sstables (like 1 TB) that won’t > compact because there are not
4 similar-sized ones able to be compacted Yes, that's exactly what's happening.
INTERNAL USE
On 2022-12-29 21:54, Durity, Sean R via user wrote:
> At some point you will end up with large sstables (like 1 TB) that won’t
> compact because there are not 4 similar-sized ones able to be compacted
Yes, that's exactly what's happening.
I'll see maybe just one more compaction, since the biggest sstable is
already more than 20% of residual free space.
> For me, the backup strategy shouldn’t drive the rest.
Mhh, yes, that makes sense.
> And if your data is ever-growing
> and never deleted, you will be adding nodes to handle the extra data as
> time goes by (and running clean-up on the existing nodes).
What will happen when adding new nodes, as you say, though?
If I have a 1GB sstable with 250GB of data that will be no longer useful
(as a new node will be the new owner) will that sstable be reduced to
750GB by "cleanup" or will it retain old data?
Thanks,
--
Lapo Luchini
mailto:lapo@lapo.it
RE: Best compaction strategy for rarely used data
Posted by "Durity, Sean R via user" <us...@cassandra.apache.org>.
Yes, clean-up will reduce the disk space on the existing nodes by re-writing only the data that the node now owns into new sstables.
Sean R. Durity
DB Solutions
Staff Systems Engineer – Cassandra
From: Lapo Luchini <la...@lapo.it>
Sent: Friday, December 30, 2022 4:12 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Best compaction strategy for rarely used data
On 2022-12-29 21: 54, Durity, Sean R via user wrote: > At some point you will end up with large sstables (like 1 TB) that won’t > compact because there are not 4 similar-sized ones able to be compacted Yes, that's exactly what's happening.
INTERNAL USE
On 2022-12-29 21:54, Durity, Sean R via user wrote:
> At some point you will end up with large sstables (like 1 TB) that won’t
> compact because there are not 4 similar-sized ones able to be compacted
Yes, that's exactly what's happening.
I'll see maybe just one more compaction, since the biggest sstable is
already more than 20% of residual free space.
> For me, the backup strategy shouldn’t drive the rest.
Mhh, yes, that makes sense.
> And if your data is ever-growing
> and never deleted, you will be adding nodes to handle the extra data as
> time goes by (and running clean-up on the existing nodes).
What will happen when adding new nodes, as you say, though?
If I have a 1GB sstable with 250GB of data that will be no longer useful
(as a new node will be the new owner) will that sstable be reduced to
750GB by "cleanup" or will it retain old data?
Thanks,
--
Lapo Luchini
lapo@lapo.it<ma...@lapo.it>
Re: Best compaction strategy for rarely used data
Posted by Lapo Luchini <la...@lapo.it>.
On 2022-12-29 21:54, Durity, Sean R via user wrote:
> At some point you will end up with large sstables (like 1 TB) that won’t
> compact because there are not 4 similar-sized ones able to be compacted
Yes, that's exactly what's happening.
I'll see maybe just one more compaction, since the biggest sstable is
already more than 20% of residual free space.
> For me, the backup strategy shouldn’t drive the rest.
Mhh, yes, that makes sense.
> And if your data is ever-growing
> and never deleted, you will be adding nodes to handle the extra data as
> time goes by (and running clean-up on the existing nodes).
What will happen when adding new nodes, as you say, though?
If I have a 1GB sstable with 250GB of data that will be no longer useful
(as a new node will be the new owner) will that sstable be reduced to
750GB by "cleanup" or will it retain old data?
Thanks,
--
Lapo Luchini
lapo@lapo.it
RE: Best compaction strategy for rarely used data
Posted by "Durity, Sean R via user" <us...@cassandra.apache.org>.
If there isn’t a TTL and timestamp on the data, I’m not sure the benefits of TWCS for this use case. I would stick with size-tiered. At some point you will end up with large sstables (like 1 TB) that won’t compact because there are not 4 similar-sized ones able to be compacted (assuming default parameters for STCS). And if your data is ever-growing and never deleted, you will be adding nodes to handle the extra data as time goes by (and running clean-up on the existing nodes). For me, the backup strategy shouldn’t drive the rest.
Sean R. Durity
From: Paul Chandler <pa...@redshots.com>
Sent: Thursday, December 29, 2022 4:51 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Best compaction strategy for rarely used data
Hi Lapo Take a look at TWCS, I think that could help your use case: https: //thelastpickle. com/blog/2016/12/08/TWCS-part1. html [thelastpickle. com] Regards Paul Chandler Sent from my iPhone On 29 Dec 2022, at 08: 55, Lapo Luchini <lapo@ lapo. it>
Hi Lapo
Take a look at TWCS, I think that could help your use case: https://thelastpickle.com/blog/2016/12/08/TWCS-part1.html [thelastpickle.com]<https://urldefense.com/v3/__https:/thelastpickle.com/blog/2016/12/08/TWCS-part1.html__;!!M-nmYVHPHQ!MthfOMR5U4-KTIvyS7qxtedGqxTx_C4S8cTa5Ym_htV3A1GA835t-aZNPYmXJexah45xutyI4Gra2ZwqSDcj$>
Regards
Paul Chandler
Sent from my iPhone
On 29 Dec 2022, at 08:55, Lapo Luchini <la...@lapo.it>> wrote:
Hi, I have a table which gets (a lot of) data that is written once and very rarely read (it is used for data that is mandatory for regulatory reasons), and almost never deleted.
I'm using the default SCTS as at the time I didn't know any better, but SSTables size are getting huge, which is a problem because they both are getting to the size of the available disk and both because I'm using a snapshot-based system to backup the node (and thus compacting a huge SSTable into an even bigger one generates a lot of traffic for mostly-old data).
I'm thinking about switching to LCS (mainly to solve the size issue), but I read that it is "optimized for read heavy workloads […] not a good choice for immutable time series data". Given that I don't really care about write nor read speed, but would like SSTables size to have a upper limit, would this strategy still be the best?
PS: Googling around a strategy called "incremental compaction" (ICS) keeps getting in results, but that's only available in ScyllaDB, right?
--
Lapo Luchini
lapo@lapo.it<ma...@lapo.it>
INTERNAL USE
Re: Best compaction strategy for rarely used data
Posted by Paul Chandler <pa...@redshots.com>.
Hi Lapo
Take a look at TWCS, I think that could help your use case: https://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
Regards
Paul Chandler
Sent from my iPhone
> On 29 Dec 2022, at 08:55, Lapo Luchini <la...@lapo.it> wrote:
>
> Hi, I have a table which gets (a lot of) data that is written once and very rarely read (it is used for data that is mandatory for regulatory reasons), and almost never deleted.
>
> I'm using the default SCTS as at the time I didn't know any better, but SSTables size are getting huge, which is a problem because they both are getting to the size of the available disk and both because I'm using a snapshot-based system to backup the node (and thus compacting a huge SSTable into an even bigger one generates a lot of traffic for mostly-old data).
>
> I'm thinking about switching to LCS (mainly to solve the size issue), but I read that it is "optimized for read heavy workloads […] not a good choice for immutable time series data". Given that I don't really care about write nor read speed, but would like SSTables size to have a upper limit, would this strategy still be the best?
>
> PS: Googling around a strategy called "incremental compaction" (ICS) keeps getting in results, but that's only available in ScyllaDB, right?
>
> --
> Lapo Luchini
> lapo@lapo.it
>