Updated sstable size for LCS, ran upgradesstables, file sizes didn't change

[Andrew Bialecki <an...@gmail.com>]

We're potentially considering increasing the size of our sstables for some
column families from 10MB to something larger.

In test, we've been trying to verify that the sstable file sizes change and
then doing a bit of benchmarking. However when we run alter the column
family and then run "nodetool upgradesstables -a keyspace columnfamily,"
the files in the data directory have been re-written, but the file sizes
are the same.

Is this the expected behavior? If not, what's the right way to upgrade
them. If this is expected, how can we benchmark the read/write performance
with varying sstable sizes.

Thanks in advance!

Andrew

Re: Updated sstable size for LCS, ran upgradesstables, file sizes didn't change

[Robert Coli <rc...@eventbrite.com>]

On Fri, Jun 21, 2013 at 4:40 PM, Andrew Bialecki
<an...@gmail.com> wrote:
> However when we run alter the column
> family and then run "nodetool upgradesstables -a keyspace columnfamily," the
> files in the data directory have been re-written, but the file sizes are the
> same.
>
> Is this the expected behavior? If not, what's the right way to upgrade them.
> If this is expected, how can we benchmark the read/write performance with
> varying sstable sizes.

It is expected, upgradesstables/scrub/clean compactions work on a
single sstable at a time, they are not capable of combining or
splitting them.

In theory you could probably :

1) start out with the largest size you want to test
2) stop your node
3) use sstable_split [1] to split sstables
4) start node, test
5) repeat 2-4

I am not sure if there is anything about level compaction which makes
this infeasible.

=Rob
[1] https://github.com/pcmanus/cassandra/tree/sstable_split

Re: Updated sstable size for LCS, ran upgradesstables, file sizes didn't change

["Hiller, Dean" <De...@nrel.gov>]

We would be very very interested in your results.  We currently run 10M but have heard of 256M sizes as well.

Please let us know what you find out.
Thanks,
Dean

From: Andrew Bialecki <an...@gmail.com>>
Reply-To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Date: Friday, June 21, 2013 5:40 PM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Updated sstable size for LCS, ran upgradesstables, file sizes didn't change

We're potentially considering increasing the size of our sstables for some column families from 10MB to something larger.

In test, we've been trying to verify that the sstable file sizes change and then doing a bit of benchmarking. However when we run alter the column family and then run "nodetool upgradesstables -a keyspace columnfamily," the files in the data directory have been re-written, but the file sizes are the same.

Is this the expected behavior? If not, what's the right way to upgrade them. If this is expected, how can we benchmark the read/write performance with varying sstable sizes.

Thanks in advance!

Andrew