You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Yatong Zhang <bl...@gmail.com> on 2014/05/04 04:21:44 UTC
Is the updating compaction strategy from 'sized tiered' to 'leveled'
automatic or need to be done manually?
Hi there,
I am updating compaction strategy from 'sized tiered' to 'leveled' and from
http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it
is said:
When updating an existing column family, reads and writes can continue as
> usual while leveling of existing sstables is performed in the background.
>
But I still see many old sstables with very large size and an old file
date. So I am wondering is the updating of compaction done automatically?
If yes, is there an estimate of time it will take? If not, what's the steps
to do it manually?
I've googled a lot but can't find something helpful. Thanks for any
feedbacks and any help is of great appreciation.
Re: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually? [heur]
Posted by Yatong Zhang <bl...@gmail.com>.
Thanks for the reply Viktor, but the whole cluster has been working all the
time and I am sure the cf got writes every second. I still don't see the
huge sstables being split into small ones
On Tue, May 6, 2014 at 5:30 PM, Viktor Jevdokimov <
Viktor.Jevdokimov@adform.com> wrote:
> Enough to write 1 column and run nodetool flush.
>
>
>
> *From:* Viktor Jevdokimov [mailto:Viktor.Jevdokimov@adform.com]
> *Sent:* Tuesday, May 6, 2014 12:00 PM
> *To:* user@cassandra.apache.org
> *Subject:* RE: Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> I mean insert/write data. When data fills memtable, memtable is flushed to
> disk as sstable, when new sstable is created, Cassandra will check if
> compaction is needed and triggers one.
>
>
>
> *From:* Yatong Zhang [mailto:blueflycn@gmail.com <bl...@gmail.com>]
> *Sent:* Monday, May 5, 2014 9:54 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> What you mean 'you need write to this CF'? I've changed the schema by
> using CQL3 'alter table' statments.
>
>
>
> On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <
> Viktor.Jevdokimov@adform.com> wrote:
>
> To trigger LCS you need to write to this CF and wait when new sstable
> flushes. I can’t find any other way to start LCS.
>
>
>
> Best regards / Pagarbiai
>
> *Viktor Jevdokimov*
>
> Senior Developer
>
>
>
> Email: Viktor.Jevdokimov@adform.com
>
> Phone: +370 5 212 3063, Fax +370 5 261 0453
>
> J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
>
> Follow us on Twitter: @adforminsider <http://twitter.com/#!/adforminsider>
>
> Experience Adform DNA <http://vimeo.com/76421547>
>
> [image: Adform News] <http://www.adform.com>
>
> [image: Adform awarded the Best Employer 2012]<http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>
>
> *From:* Yatong Zhang [mailto:blueflycn@gmail.com]
> *Sent:* Sunday, May 4, 2014 5:22 AM
> *To:* user@cassandra.apache.org
> *Subject:* Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> Hi there,
>
> I am updating compaction strategy from 'sized tiered' to 'leveled' and
> from
> http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandrait is said:
>
> When updating an existing column family, reads and writes can continue as
> usual while leveling of existing sstables is performed in the background.
>
>
>
> But I still see many old sstables with very large size and an old file
> date. So I am wondering is the updating of compaction done automatically?
> If yes, is there an estimate of time it will take? If not, what's the steps
> to do it manually?
>
> I've googled a lot but can't find something helpful. Thanks for any
> feedbacks and any help is of great appreciation.
>
>
>
RE: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually? [heur]
Posted by Viktor Jevdokimov <Vi...@adform.com>.
Enough to write 1 column and run nodetool flush.
From: Viktor Jevdokimov [mailto:Viktor.Jevdokimov@adform.com]
Sent: Tuesday, May 6, 2014 12:00 PM
To: user@cassandra.apache.org
Subject: RE: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
I mean insert/write data. When data fills memtable, memtable is flushed to disk as sstable, when new sstable is created, Cassandra will check if compaction is needed and triggers one.
From: Yatong Zhang [mailto:blueflycn@gmail.com]
Sent: Monday, May 5, 2014 9:54 AM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
What you mean 'you need write to this CF'? I've changed the schema by using CQL3 'alter table' statments.
On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <Vi...@adform.com>> wrote:
To trigger LCS you need to write to this CF and wait when new sstable flushes. I can’t find any other way to start LCS.
Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com<ma...@adform.com>
Phone: +370 5 212 3063<tel:%2B370%205%20212%203063>, Fax +370 5 261 0453<tel:%2B370%205%20261%200453>
J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
Follow us on Twitter: @adforminsider<http://twitter.com/#!/adforminsider>
Experience Adform DNA<http://vimeo.com/76421547>
[Adform News]<http://www.adform.com>
[Adform awarded the Best Employer 2012]<http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
From: Yatong Zhang [mailto:blueflycn@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 4, 2014 5:22 AM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
Hi there,
I am updating compaction strategy from 'sized tiered' to 'leveled' and from http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it is said:
When updating an existing column family, reads and writes can continue as usual while leveling of existing sstables is performed in the background.
But I still see many old sstables with very large size and an old file date. So I am wondering is the updating of compaction done automatically? If yes, is there an estimate of time it will take? If not, what's the steps to do it manually?
I've googled a lot but can't find something helpful. Thanks for any feedbacks and any help is of great appreciation.
RE: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually? [heur]
Posted by Viktor Jevdokimov <Vi...@adform.com>.
I mean insert/write data. When data fills memtable, memtable is flushed to disk as sstable, when new sstable is created, Cassandra will check if compaction is needed and triggers one.
From: Yatong Zhang [mailto:blueflycn@gmail.com]
Sent: Monday, May 5, 2014 9:54 AM
To: user@cassandra.apache.org
Subject: Re: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
What you mean 'you need write to this CF'? I've changed the schema by using CQL3 'alter table' statments.
On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <Vi...@adform.com>> wrote:
To trigger LCS you need to write to this CF and wait when new sstable flushes. I can’t find any other way to start LCS.
Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com<ma...@adform.com>
Phone: +370 5 212 3063<tel:%2B370%205%20212%203063>, Fax +370 5 261 0453<tel:%2B370%205%20261%200453>
J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
Follow us on Twitter: @adforminsider<http://twitter.com/#!/adforminsider>
Experience Adform DNA<http://vimeo.com/76421547>
[Adform News]<http://www.adform.com>
[Adform awarded the Best Employer 2012]<http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
From: Yatong Zhang [mailto:blueflycn@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 4, 2014 5:22 AM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
Hi there,
I am updating compaction strategy from 'sized tiered' to 'leveled' and from http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it is said:
When updating an existing column family, reads and writes can continue as usual while leveling of existing sstables is performed in the background.
But I still see many old sstables with very large size and an old file date. So I am wondering is the updating of compaction done automatically? If yes, is there an estimate of time it will take? If not, what's the steps to do it manually?
I've googled a lot but can't find something helpful. Thanks for any feedbacks and any help is of great appreciation.
Re: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually?
Posted by Yatong Zhang <bl...@gmail.com>.
I tried to run 'nodetool compact' (or with keyspace and cfname), seemed not
woking. The command hung there and nothing happened. But I noticed that
there was about more 30000+ pending tasks when using 'nodetool
compactionsats'
On Tue, May 6, 2014 at 1:45 AM, Robert Coli <rc...@eventbrite.com> wrote:
> On Mon, May 5, 2014 at 12:24 AM, Daniel Chia <da...@coursera.org> wrote:
>
>> If you want to trigger a conversion to LCS of the on-disk sstables, doing
>> "nodetool compact <cf>" should achieve what you want.
>>
>
> For more detail and a caveat :
>
> https://issues.apache.org/jira/browse/CASSANDRA-6092
> "
> The workaround to call "nodetool compact" only works when there is at
> least some activity, causing there to be more than 1 sstable. I've just
> tested with a cluster that was not receiving any reads nor writes. In that
> case calling "nodetool compact" does not do anything ...
> "
>
> =Rob
>
>
Re: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually?
Posted by Robert Coli <rc...@eventbrite.com>.
On Mon, May 5, 2014 at 12:24 AM, Daniel Chia <da...@coursera.org> wrote:
> If you want to trigger a conversion to LCS of the on-disk sstables, doing
> "nodetool compact <cf>" should achieve what you want.
>
For more detail and a caveat :
https://issues.apache.org/jira/browse/CASSANDRA-6092
"
The workaround to call "nodetool compact" only works when there is at least
some activity, causing there to be more than 1 sstable. I've just tested
with a cluster that was not receiving any reads nor writes. In that case
calling "nodetool compact" does not do anything ...
"
=Rob
Re: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually?
Posted by Daniel Chia <da...@coursera.org>.
If you want to trigger a conversion to LCS of the on-disk sstables, doing
"nodetool compact <cf>" should achieve what you want.
Be warned, if you have a lot of data in the CF, this could potentially take
a while depending on your compaction throttling.
Thanks,
Daniel
On Sun, May 4, 2014 at 11:54 PM, Yatong Zhang <bl...@gmail.com> wrote:
> What you mean 'you need write to this CF'? I've changed the schema by
> using CQL3 'alter table' statments.
>
>
> On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <
> Viktor.Jevdokimov@adform.com> wrote:
>
>> To trigger LCS you need to write to this CF and wait when new sstable
>> flushes. I can’t find any other way to start LCS.
>>
>>
>> Best regards / Pagarbiai
>> *Viktor Jevdokimov*
>> Senior Developer
>>
>> Email: Viktor.Jevdokimov@adform.com
>> Phone: +370 5 212 3063, Fax +370 5 261 0453
>> J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
>> Follow us on Twitter: @adforminsider<http://twitter.com/#!/adforminsider>
>> Experience Adform DNA <http://vimeo.com/76421547>
>> [image: Adform News] <http://www.adform.com>
>> [image: Adform awarded the Best Employer 2012]
>> <http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
>>
>> Disclaimer: The information contained in this message and attachments is
>> intended solely for the attention and use of the named addressee and may be
>> confidential. If you are not the intended recipient, you are reminded that
>> the information remains the property of the sender. You must not use,
>> disclose, distribute, copy, print or rely on this e-mail. If you have
>> received this message in error, please contact the sender immediately and
>> irrevocably delete this message and any copies.
>>
>> *From:* Yatong Zhang [mailto:blueflycn@gmail.com]
>> *Sent:* Sunday, May 4, 2014 5:22 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* Is the updating compaction strategy from 'sized tiered' to
>> 'leveled' automatic or need to be done manually? [heur]
>>
>>
>>
>> Hi there,
>>
>> I am updating compaction strategy from 'sized tiered' to 'leveled' and
>> from
>> http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandrait is said:
>>
>> When updating an existing column family, reads and writes can continue as
>> usual while leveling of existing sstables is performed in the background.
>>
>>
>>
>> But I still see many old sstables with very large size and an old file
>> date. So I am wondering is the updating of compaction done automatically?
>> If yes, is there an estimate of time it will take? If not, what's the steps
>> to do it manually?
>>
>> I've googled a lot but can't find something helpful. Thanks for any
>> feedbacks and any help is of great appreciation.
>>
>
>
Re: Is the updating compaction strategy from 'sized tiered' to
'leveled' automatic or need to be done manually?
Posted by Yatong Zhang <bl...@gmail.com>.
What you mean 'you need write to this CF'? I've changed the schema by using
CQL3 'alter table' statments.
On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <
Viktor.Jevdokimov@adform.com> wrote:
> To trigger LCS you need to write to this CF and wait when new sstable
> flushes. I can’t find any other way to start LCS.
>
>
> Best regards / Pagarbiai
> *Viktor Jevdokimov*
> Senior Developer
>
> Email: Viktor.Jevdokimov@adform.com
> Phone: +370 5 212 3063, Fax +370 5 261 0453
> J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
> Follow us on Twitter: @adforminsider <http://twitter.com/#!/adforminsider>
> Experience Adform DNA <http://vimeo.com/76421547>
> [image: Adform News] <http://www.adform.com>
> [image: Adform awarded the Best Employer 2012]
> <http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
> *From:* Yatong Zhang [mailto:blueflycn@gmail.com]
> *Sent:* Sunday, May 4, 2014 5:22 AM
> *To:* user@cassandra.apache.org
> *Subject:* Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> Hi there,
>
> I am updating compaction strategy from 'sized tiered' to 'leveled' and
> from
> http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandrait is said:
>
> When updating an existing column family, reads and writes can continue as
> usual while leveling of existing sstables is performed in the background.
>
>
>
> But I still see many old sstables with very large size and an old file
> date. So I am wondering is the updating of compaction done automatically?
> If yes, is there an estimate of time it will take? If not, what's the steps
> to do it manually?
>
> I've googled a lot but can't find something helpful. Thanks for any
> feedbacks and any help is of great appreciation.
>
RE: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually?
Posted by Viktor Jevdokimov <Vi...@adform.com>.
To trigger LCS you need to write to this CF and wait when new sstable flushes. I can’t find any other way to start LCS.
Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com<ma...@adform.com>
Phone: +370 5 212 3063, Fax +370 5 261 0453
J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
Follow us on Twitter: @adforminsider<http://twitter.com/#!/adforminsider>
Experience Adform DNA<http://vimeo.com/76421547>
[Adform News] <http://www.adform.com>
[Adform awarded the Best Employer 2012] <http://www.adform.com/site/blog/adform/adform-takes-top-spot-in-best-employer-survey/>
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
From: Yatong Zhang [mailto:blueflycn@gmail.com]
Sent: Sunday, May 4, 2014 5:22 AM
To: user@cassandra.apache.org
Subject: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]
Hi there,
I am updating compaction strategy from 'sized tiered' to 'leveled' and from http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it is said:
When updating an existing column family, reads and writes can continue as usual while leveling of existing sstables is performed in the background.
But I still see many old sstables with very large size and an old file date. So I am wondering is the updating of compaction done automatically? If yes, is there an estimate of time it will take? If not, what's the steps to do it manually?
I've googled a lot but can't find something helpful. Thanks for any feedbacks and any help is of great appreciation.