You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Maki Watanabe (Created) (JIRA)" <ji...@apache.org> on 2012/03/02 04:23:09 UTC
[jira] [Created] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
----------------------------------------------------------------------------------------------
Key: CASSANDRA-3989
URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.7
Environment: RHEL6
Reporter: Maki Watanabe
Priority: Minor
1.0.7 + LeveledCompactionStrategy
If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
Reproduce procedure:
# create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
# Insert some data into CF1.
# nodetool flush
# Verify the sstable is created at L1 in CF1.json
# nodetool cleanup
# Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
# repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224984#comment-13224984 ]
Peter Schuller commented on CASSANDRA-3989:
-------------------------------------------
Sorry, I think I just pulled before merge-to-trunk and was still looking at a trunk containing the original version of the patch.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch, 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maki Watanabe updated CASSANDRA-3989:
-------------------------------------
Comment: was deleted
(was: Fix LeveledManifest.promote not to promote sstables to next level if the number of source sstables (removed) is 1.)
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Environment: RHEL6
> Reporter: Maki Watanabe
> Priority: Minor
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maki Watanabe updated CASSANDRA-3989:
-------------------------------------
Attachment: (was: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch)
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229870#comment-13229870 ]
Maki Watanabe commented on CASSANDRA-3989:
------------------------------------------
Removed first patch file I attached here because it is harmful.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maki Watanabe updated CASSANDRA-3989:
-------------------------------------
Attachment: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
Fix LeveledManifest.promote will not promote sstables to next level if the number of source sstables (removed) is one.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Environment: RHEL6
> Reporter: Maki Watanabe
> Priority: Minor
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224509#comment-13224509 ]
Jonathan Ellis commented on CASSANDRA-3989:
-------------------------------------------
+1
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch, 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224001#comment-13224001 ]
Maki Watanabe edited comment on CASSANDRA-3989 at 3/7/12 5:56 AM:
------------------------------------------------------------------
It is more complicated than I first thought.
getCandidatesFor(int level) returns next compaction candidates for the level by:
- sort the generations and find a sstable where we left off last time
- and then find overlappng sstables from next level
So if we add a new sstable into same level, getCompactionCandidates won't return empty, and then getNextBackgoundTask returns non-null task forever.
I think we should better to back out the fix and look for better strategy to resolve the issue.
was (Author: makiw):
It is more complicated than I first thought.
getCandidatesFor(int level) returns next compaction candidates for the level by:
- sor the generations and find a sstable where we left off last time
- and then find overlappng sstables from next level
So if we add a new sstable into same level, getCompactionCandidates won't return empty, and then getNextBackgoundTask returns non-null task forever.
I think we should better to back out the fix and look for better strategy to resolve the issue.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224001#comment-13224001 ]
Maki Watanabe commented on CASSANDRA-3989:
------------------------------------------
It is more complicated than I first thought.
getCandidatesFor(int level) returns next compaction candidates for the level by:
- sor the generations and find a sstable where we left off last time
- and then find overlappng sstables from next level
So if we add a new sstable into same level, getCompactionCandidates won't return empty, and then getNextBackgoundTask returns non-null task forever.
I think we should better to back out the fix and look for better strategy to resolve the issue.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Sylvain Lebresne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-3989:
----------------------------------------
Attachment: 3989.txt
Attaching another approach that makes the compaction type available to the leveled compaction that for cleanup, scrub and upgradeSSTables simply replace the old sstable by the new one (without changing level or anything else). The rational is those operation don't really "change" the sstable content and should simply "replace" sstables.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch, 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224979#comment-13224979 ]
Peter Schuller commented on CASSANDRA-3989:
-------------------------------------------
Hmm, Does this address the case from CASSANDRA-3952?
{quote}
I think this might happen whenever there is exactly one sstable in L0 that is large enough for the score for L0 to be > 1, and if L1 is full (so that skipLevels doesn't promote).
It's possible the sstables I had lying around from sized-tiered compaction combined with my write load conspired to trigger this case.
{quote}
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch, 3989.txt
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223924#comment-13223924 ]
Maki Watanabe commented on CASSANDRA-3989:
------------------------------------------
Another problem with the fix.
In following condition, the background compaction task start looping.
- No L0 sstable
- Enough number of L1 sstables, exceed L1 capacity (compaction score > 1.1)
It seems the background task try to promote sstables to reduce compaction score, but it can't by this fix, then it will compact each L1 sstables forever.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Sylvain Lebresne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224324#comment-13224324 ]
Sylvain Lebresne commented on CASSANDRA-3989:
---------------------------------------------
I agree, the committed fix does more harm than it helps so I reverted it.
I'm trying some other approach so will hopefully attach another version soon.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222995#comment-13222995 ]
Maki Watanabe commented on CASSANDRA-3989:
------------------------------------------
Johathan,
This fix has a problem. If you have only one Level 0 sstable, and there are no Level 1 sstables involved in the compaction, promote method will chooses 0 as newLevel, and it causes assertion.
Should I create a new ticket?
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Maki Watanabe (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223924#comment-13223924 ]
Maki Watanabe edited comment on CASSANDRA-3989 at 3/7/12 2:48 AM:
------------------------------------------------------------------
Another problem with the fix.
In following condition, the background compaction task start looping.
- No L0 sstable
- Enough number of L1 sstables, exceed L1 capacity (compaction score > 1.1)
It seems the background task try to promote sstables to reduce compaction score, but it can't by this fix, then it will compact each L1 sstables forever.
was (Author: makiw):
Another problem with the fix.
In following condition, the background compaction task start looping.
- No L0 sstable
- Enough number of L1 sstables, exceed L1 capacity (compaction score > 1.1)
It seems the background task try to promote sstables to reduce compaction score, but it can't by this fix, then it will compact each L1 sstables forever.
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3989:
--------------------------------------
Affects Version/s: (was: 1.0.7)
1.0.0
Fix Version/s: 1.1.0
1.0.9
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-3989) nodetool
cleanup/scrub/upgradesstables promotes all sstables to next level
(LeveledCompaction)
Posted by "Jonathan Ellis (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reopened CASSANDRA-3989:
---------------------------------------
No, let's reopen this one
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-3989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: RHEL6
> Reporter: Maki Watanabe
> Assignee: Maki Watanabe
> Priority: Minor
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction for each sstable. During the compaction, it put the new sstable to next level of the original sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608 will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb: 5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira