You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Aaron Morton (Created) (JIRA)" <ji...@apache.org> on 2012/03/01 11:39:59 UTC
[jira] [Created] (CASSANDRA-3985) Always ensure enough space for
Compaction
Always ensure enough space for Compaction
-----------------------------------------
Key: CASSANDRA-3985
URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.7
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Minor
>From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
The result is a NPE, partial stack
{code:java}
$ nodetool -h localhost upgradesstables
Error occured while upgrading the sstables for keyspace MyKeySpace
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
<snip>
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:222)
at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
{code}
(night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Aaron Morton (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243834#comment-13243834 ]
Aaron Morton commented on CASSANDRA-3985:
-----------------------------------------
Sorry for not getting back.
Thanks.
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224484#comment-13224484 ]
Pavel Yaskevich commented on CASSANDRA-3985:
--------------------------------------------
Question - what is the reason why we only return "true" from "ensureFreeSpace" if action is not user defined?
Few styling issues:
{code}
public synchronized static String getDataFileLocationForTable(String table, long expectedCompactedFileSize,
boolean ensureFreeSpace )
{code}
should be changed to
{code}
public synchronized static String getDataFileLocationForTable(String table,
long expectedCompactedFileSize,
boolean ensureFreeSpace)
{code}
or all arguments written on the same line.
Also we don't use spaces to delimit operands e.g.
{code}
for ( int i = 0 ; i < dataDirectoryForTable.length ; i++ )
{code}
I can see those styling problems inside of getDataFileLocationForTable(...) method.
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267889#comment-13267889 ]
Jonathan Ellis commented on CASSANDRA-3985:
-------------------------------------------
I don't understand this. This will still return null for user defined compactions under the same conditions it would have before, with no log message, since ensureFreeSpace == !isUserDefined.
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-3985.
---------------------------------------
Resolution: Fixed
Fix Version/s: (was: 1.0.9)
1.1.1
1.0.10
Assignee: Jonathan Ellis (was: Aaron Morton)
committed
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.0.10, 1.1.1
>
> Attachments: 3985-2.txt, cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267893#comment-13267893 ]
Jonathan Ellis commented on CASSANDRA-3985:
-------------------------------------------
(All the other callers of {{getDataFileLocation}} and {{getDataFileLocationForTable}} already check for null correctly.)
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: 3985-2.txt, cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Aaron Morton (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Morton updated CASSANDRA-3985:
------------------------------------
Summary: Ensure a directory is selected for Compaction (was: Always ensure enough space for Compaction)
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reopened CASSANDRA-3985:
---------------------------------------
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3985:
--------------------------------------
Attachment: 3985-2.txt
It seems to me that the real fix we need is to turn the assert in CompactionTask into an if statement so it can't be turned off. Patch attached.
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: 3985-2.txt, cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267911#comment-13267911 ]
Pavel Yaskevich commented on CASSANDRA-3985:
--------------------------------------------
+1, right now we won't try to return the directory with the biggest empty space and would abort compaction if there is no sufficient space left.
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: 3985-2.txt, cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3985:
--------------------------------------
Reviewer: xedin
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3985) Ensure a directory is selected
for Compaction
Posted by "Aaron Morton (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Morton updated CASSANDRA-3985:
------------------------------------
Attachment: cassandra-1.0-3985.txt
Modified DatabaseDescriptor.getDataFileLocationForTable() to add ensureFreeSpace.
Reformatted getDataFileLocationForTable() as it was small.
Removed unused DatabaseDescriptor.getNextAvailableDataLocation()
> Ensure a directory is selected for Compaction
> ---------------------------------------------
>
> Key: CASSANDRA-3985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3985
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Attachments: cassandra-1.0-3985.txt
>
>
> From http://www.mail-archive.com/user@cassandra.apache.org/msg20757.html
> CompactionTask.execute() checks if there is a valid compactionFileLocation only if partialCompactionsAcceptable() . upgradesstables results in a CompactionTask with userdefined set, so the valid location check is not performed.
> The result is a NPE, partial stack
> {code:java}
> $ nodetool -h localhost upgradesstables
> Error occured while upgrading the sstables for keyspace MyKeySpace
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
> at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219)
> at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:995)
> at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1648)
> <snip>
> Caused by: java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:641)
> at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:652)
> at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:1888)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:151)
> at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:229)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:182)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> (night time here, will fix tomorrow, anyone else feel free to fix it.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira