You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Paul Querna (JIRA)" <ji...@apache.org> on 2010/12/03 01:34:10 UTC
[jira] Created: (CASSANDRA-1811) Cleanup smallest CFs first
Cleanup smallest CFs first
--------------------------
Key: CASSANDRA-1811
URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Paul Querna
Priority: Minor
When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Hermes updated CASSANDRA-1811:
----------------------------------
Attachment: 1811.txt
This may need to be rebased if 1812 is committed first and vice-versa.
Oh well.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.6.8
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
> Attachments: 1811.txt
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969772#action_12969772 ]
Jon Hermes commented on CASSANDRA-1811:
---------------------------------------
Yes.
v2 +1.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.6.8
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
> Attachments: 1811-v2.txt, 1811.txt
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1811:
--------------------------------------
Fix Version/s: 0.6.9
Assignee: Jon Hermes
Good idea. We'll get this in 0.6.9.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1811:
--------------------------------------
Attachment: 1811-v2.txt
Isn't it a lot simpler to (a) just sort a List instead of going through a TreeSet and (b) base it on columnFamilyStores.values() instead of metadata + get for each one?
v2 attached.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.6.8
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
> Attachments: 1811-v2.txt, 1811.txt
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1811) Cleanup smallest CFs
first
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966751#action_12966751 ]
Jon Hermes edited comment on CASSANDRA-1811 at 12/3/10 7:56 PM:
----------------------------------------------------------------
-This may need to be rebased if 1812 is committed first and vice-versa.-
-Oh well.-
Nope, both apply cleanly. Hurrah!
was (Author: jhermes):
This may need to be rebased if 1812 is committed first and vice-versa.
Oh well.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.6.8
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
> Attachments: 1811.txt
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Paul Querna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966362#action_12966362 ]
Paul Querna commented on CASSANDRA-1811:
----------------------------------------
An alternative would be to extend forceclean in jmx to allow you to specify both the Keyspace and a CF, so that you could manually do specific CFs.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Paul Querna
> Priority: Minor
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966706#action_12966706 ]
Jon Hermes commented on CASSANDRA-1811:
---------------------------------------
bq. An alternative would be to extend forceclean in jmx to allow you to specify both the Keyspace and a CF, so that you could manually do specific CFs.
This is resolved in CASSANDRA-1812, but there's value in both.
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1811) Cleanup smallest CFs first
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969929#action_12969929 ]
Hudson commented on CASSANDRA-1811:
-----------------------------------
Integrated in Cassandra-0.6 #21 (See [https://hudson.apache.org/hudson/job/Cassandra-0.6/21/])
cleanup smallest CFs first to increase free temp space for larger ones
patch by Jon Hermes and jbellis for CASSANDRA-1811
> Cleanup smallest CFs first
> --------------------------
>
> Key: CASSANDRA-1811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1811
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.6.8
> Reporter: Paul Querna
> Assignee: Jon Hermes
> Priority: Minor
> Fix For: 0.6.9, 0.7.0
>
> Attachments: 1811-v2.txt, 1811.txt
>
>
> When running a cleanup, it would be an advantage to anti-compact the smallest SSTables first, so that free disk space is gradually increased, so that larger sstables later on are more likely to successfully anti-compact.
> In 0.6, currently Table.forceCleanup() just iterates the list of CFs in whatever order they come from tableMetadata.getColumnFamilies, which is just a keySet().
> The code should be changed to sort the CFs, smallest first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.