You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Cheng Zhang (JIRA)" <ji...@apache.org> on 2012/07/27 18:53:33 UTC
[jira] [Created] (CASSANDRA-4467) insufficient space for compaction
when upgrade compaction strategy from SizeTiered to Leveled
Cheng Zhang created CASSANDRA-4467:
--------------------------------------
Summary: insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
Key: CASSANDRA-4467
URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.10
Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
Reporter: Cheng Zhang
Cassandra has two data directories as follow:
cassandra-disk0 use about 500G, about 250G free space
cassandra-disk1 use about 500G, about 250G free space
The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4467) insufficient space for compaction
when upgrade compaction strategy from SizeTiered to Leveled
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4467:
--------------------------------------
Priority: Minor (was: Major)
Affects Version/s: (was: 1.0.10)
1.0.0
Assignee: Yuki Morishita
Yuki, can you have a look at what would be required to make compaction able to span multiple target directories?
My gut is that this is not a small change, in which case I'd rather mark it Later since it's a pretty niche scenario.
> insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.0
> Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
> Reporter: Cheng Zhang
> Assignee: Yuki Morishita
> Priority: Minor
>
> Cassandra has two data directories as follow:
> cassandra-disk0 use about 500G, about 250G free space
> cassandra-disk1 use about 500G, about 250G free space
> The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4467) insufficient space for compaction
when upgrade compaction strategy from SizeTiered to Leveled
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4467:
--------------------------------------
Reviewer: jbellis
Fix Version/s: 1.2.1
> insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.0
> Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
> Reporter: Cheng Zhang
> Assignee: Jason Brown
> Priority: Minor
> Fix For: 1.2.1
>
>
> Cassandra has two data directories as follow:
> cassandra-disk0 use about 500G, about 250G free space
> cassandra-disk1 use about 500G, about 250G free space
> The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4467) insufficient space for compaction
when upgrade compaction strategy from SizeTiered to Leveled
Posted by "Cheng Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Zhang updated CASSANDRA-4467:
-----------------------------------
Issue Type: Improvement (was: Bug)
> insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.10
> Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
> Reporter: Cheng Zhang
>
> Cassandra has two data directories as follow:
> cassandra-disk0 use about 500G, about 250G free space
> cassandra-disk1 use about 500G, about 250G free space
> The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4467) insufficient space for
compaction when upgrade compaction strategy from SizeTiered to Leveled
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-4467:
-----------------------------------------
Assignee: Jason Brown (was: Yuki Morishita)
Maybe one simple improvement would could make would be to have LeveledManifest.getCandidatesFor be disk-space aware and stop collecting L0 candidates before output space is exhausted.
> insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.0
> Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
> Reporter: Cheng Zhang
> Assignee: Jason Brown
> Priority: Minor
> Fix For: 1.2.1
>
>
> Cassandra has two data directories as follow:
> cassandra-disk0 use about 500G, about 250G free space
> cassandra-disk1 use about 500G, about 250G free space
> The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4467) insufficient space for
compaction when upgrade compaction strategy from SizeTiered to Leveled
Posted by "Vitalii Tymchyshyn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450484#comment-13450484 ]
Vitalii Tymchyshyn commented on CASSANDRA-4467:
-----------------------------------------------
I did also hit this. Upgrade requires double the space. As for me, workaround was to move large files out of data directory, upgrade, then move them back to be picked up.
I'd say LeveledCompaction upgrade can do the same - convert not all at once, but as much as it can multiple times. It will be slower, but you won't have this:
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,600 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,600 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,600 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,601 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,601 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,601 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,601 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,602 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,602 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,602 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,602 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,602 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,603 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
WARN [CompactionExecutor:17] 2012-09-07 12:09:47,603 CompactionTask.java (line 94) insufficient space to compact; aborting compaction
BTW: When I did return files and asked to discover new files, It've also picked up some files current compaction run was producing. Dunno if it's OK.
> insufficient space for compaction when upgrade compaction strategy from SizeTiered to Leveled
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4467
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.0
> Environment: Ubuntu,Oracle Java 1.7,Cassandra 1.0.10
> Reporter: Cheng Zhang
> Assignee: Yuki Morishita
> Priority: Minor
>
> Cassandra has two data directories as follow:
> cassandra-disk0 use about 500G, about 250G free space
> cassandra-disk1 use about 500G, about 250G free space
> The max data file is about 400G. When I upgrade from SizeTieredCompactionStrategy to LeveledCompactionStrategy, there is no space to do this, for the free space of every data directory is small than the largest data file. But the total free space is enough for compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira