You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Benjamin Coverston (JIRA)" <ji...@apache.org> on 2011/06/14 22:16:47 UTC
[jira] [Created] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Cannot Create Duplicate Compaction Marker
-----------------------------------------
Key: CASSANDRA-2769
URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Benjamin Coverston
Concurrent compaction can trigger the following exception: I'll attach a patch to fix this.
java.io.IOError: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051130#comment-13051130 ]
Hudson commented on CASSANDRA-2769:
-----------------------------------
Integrated in Cassandra #931 (See [https://builds.apache.org/job/Cassandra/931/])
Fix compaction of the same sstable by multiple thread
patch by slebresne; reviewed by jbellis for CASSANDRA-2769
slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1136904
Files :
* /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
* /cassandra/trunk/src/java/org/apache/cassandra/db/DataTracker.java
* /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
* /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/AbstractCompactionTask.java
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2, 1.0
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables-v2.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com-v2.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049919#comment-13049919 ]
Jonathan Ellis commented on CASSANDRA-2769:
-------------------------------------------
the 0.8 patch looks good. (i did notice that some of the other post-markCompactiong code checks for null or empty, others just check for null -- one of these is probably wrong.)
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.1, 1.0
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-2769:
----------------------------------------
Attachment: 0002-Only-compact-what-has-been-succesfully-marked-as-com-v2.patch
0001-Do-compact-only-smallerSSTables-v2.patch
bq. For trunk patches, I'm not comfortable w/ 0001 reassigning the sstables field on general principles either. We could have the compaction proceed using smallerSSTables as a simpler alternative, but in general this organization feels like negative progress from the 0.8 doCompaction/doCompactionWithoutSizeEstimation.
Attaching v2 that doesn't reassign the sstables field.
bq. I think Alan has a good point. I don't think it's an appropriate role of the data tracker to modify the set of sstables to be compacted in a task.
I do not disagree with that. However I'd like that we fix trunk as a first priority. It's a pain to work on other issues (CASSANDRA-2521 for instance) while it is broken (and the goal must be to do our best to always have a working trunk). The attached patches doesn't really change any behavior, it just fixes the bugs, so let's get that in first before thinking about refactoring.
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables-v2.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com-v2.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2769) Cannot Create
Duplicate Compaction Marker
Posted by "Benjamin Coverston (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049614#comment-13049614 ]
Benjamin Coverston edited comment on CASSANDRA-2769 at 6/15/11 5:37 AM:
------------------------------------------------------------------------
Yes, in both.
was (Author: bcoverston):
Yes
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050579#comment-13050579 ]
Jonathan Ellis commented on CASSANDRA-2769:
-------------------------------------------
For trunk patches, I'm not comfortable w/ 0001 reassigning the sstables field on general principles either. We could have the compaction proceed using smallerSSTables as a simpler alternative, but in general this organization feels like negative progress from the 0.8 doCompaction/doCompactionWithoutSizeEstimation.
trunk 0002 looks fine.
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Benjamin Coverston (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049614#comment-13049614 ]
Benjamin Coverston commented on CASSANDRA-2769:
-----------------------------------------------
Yes
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050864#comment-13050864 ]
Hudson commented on CASSANDRA-2769:
-----------------------------------
Integrated in Cassandra-0.8 #173 (See [https://builds.apache.org/job/Cassandra-0.8/173/])
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049923#comment-13049923 ]
Sylvain Lebresne commented on CASSANDRA-2769:
---------------------------------------------
Alright, I've committed the 0.8 patch. I'll have a look at the checks.
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.1, 1.0
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051113#comment-13051113 ]
Jonathan Ellis commented on CASSANDRA-2769:
-------------------------------------------
+1 v2
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables-v2.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com-v2.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Alan Liang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050081#comment-13050081 ]
Alan Liang commented on CASSANDRA-2769:
---------------------------------------
Instead of letting DataTracker#markCompacting modify the subset of sstables to be compacted, I think it might be cleaner if it didn't and relied on the CompactionStrategy to select the correct sstables. We can do this by having the CompactionStrategy get the non compacting sstables from the DataTracker and work with those to generate the buckets. The strategy should also be responsible for creating buckets that fit within the min/max thresholds. #markCompacting would then be changed such that it can either accept/reject a bucket to be compacted instead of modifying the subset. #markCompacting will also serve to handle the race condition of the DataTracker being inaccurate, whereby, it will move on to other buckets.
With this, we can avoid generating buckets that are already compacting and it gives full control of what actually is compacted by the CompactionStrategy.
What do you guys think?
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049613#comment-13049613 ]
Stu Hood commented on CASSANDRA-2769:
-------------------------------------
Is this in trunk, or in 0.8.0?
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2769:
--------------------------------------
Description:
Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
java.io.IOError: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
... 12 more
was:
Concurrent compaction can trigger the following exception: I'll attach a patch to fix this.
java.io.IOError: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Unable to create compaction marker
at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
... 12 more
Assignee: Sylvain Lebresne
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Benjamin Coverston (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051086#comment-13051086 ]
Benjamin Coverston commented on CASSANDRA-2769:
-----------------------------------------------
I'm sorry I didn't mean to imply it should be fixed _here_. I'll find a more appropriate venue to vent these frustrations :)
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables-v2.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com-v2.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-2769:
----------------------------------------
Attachment: 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
0001-Do-compact-only-smallerSSTables.patch
0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch
Alright, there is a bunch of problems, one of which affects 0.8 and trunk and could cause this stackTrace. The others are due to CASSANDRA-1610 and thus only affect trunk (but one of those can also result in the attached stackTrace).
The problem affecting 0.8 and trunk is related to a left over line in doCleanup() that is wrongly unmarking a sstable from the compacting set before having removed it from the active set of sstables. Thus another compaction could start compacting this sstable and we'll end up marking the file as compacted twice (and we would have duplicated the sstable, which is a problem for counters).
Patch 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch removes it and is against 0.8.
Trunk has a few problems of its own:
* If disk space is not sufficient to compact all sstables, it computes the smallestSSTables set that fits, but doesn't use it. Attached first patch (0001-Do-compact-only-smallerSSTables.patch) fixes that.
* The CompactionTask logic wrongly decorrelates the set of sstables that are successfully marked from the ones it did compact. That is, it grabs a list of sstables it wants to compact, then call markCompacting on them, but does not check if all of them are successfully marked and compact the original list instead.
In effect, a task will recompact sstables that are already being compacted by other task and the given file will be compacted twice (or more) and marked compacted multiple times.
Attached patch (0002-Only-compact-what-has-been-succesfully-marked-as-com.patch) fixes this by changing the sstables set of a given CompactionTask to whatever has been successfully marked only. Since the marking involves updating the task, I've move the logic to AbstractCompactionTask where it seems to make more sense to me.
* For some reason, the markCompacting added for CompactionTasks was refusing to mark (and compact) anything if the set of sstable was bigger that MaxCompactionThreshold. This means that as soon as the number of sstables (of same size) in the column family would exceed the threshold, no compaction would be started. This is not the expected behavior. The second patch also fixes this by reusing the original markCompacting that handles this correctly.
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.1, 1.0
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate
Compaction Marker
Posted by "Benjamin Coverston (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050890#comment-13050890 ]
Benjamin Coverston commented on CASSANDRA-2769:
-----------------------------------------------
I think Alan has a good point. I don't think it's an appropriate role of the data tracker to modify the set of sstables to be compacted in a task. It's been a bit of a struggle to hack around the behavior of the DataTracker to ensure my compactions happen in the order I want with the SSTables that I want. That should probably be the exclusive domain compaction strategy.
> Cannot Create Duplicate Compaction Marker
> -----------------------------------------
>
> Key: CASSANDRA-2769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2769
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Benjamin Coverston
> Assignee: Sylvain Lebresne
> Fix For: 0.8.2
>
> Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch
>
>
> Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully.
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Unable to create compaction marker
> at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634)
> ... 12 more
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira