You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Daning <da...@netseer.com> on 2011/09/21 19:44:42 UTC
Unable to create compaction marker
I got this exception in system.log several times a day, do you have idea
what caused this problem and what is the consequence?
ERROR [CompactionExecutor:12] 2011-09-15 14:29:19,578
AbstractCassandraDaemon.java (line 139) Fatal exception in thread
Thread[CompactionExecutor:12,1,main]
java.io.IOError: java.io.IOException: Unable to create compaction marker
at
org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:642)
at
org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:280)
at
org.apache.cassandra.db.DataTracker.replace(DataTracker.java:253)
at
org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:214)
at
org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1094)
at
org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:599)
at
org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:507)
at
org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:142)
at
org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:108)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Unable to create compaction marker
at
org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
... 13 more
Thank you in advance.
Daning
Re: Unable to create compaction marker
Posted by Radim Kolar <hs...@sendmail.cz>.
Dne 21.9.2011 20:01, Jonathan Ellis napsal(a):
> Means Cassandra couldn't create an empty file in the data directory
> designating a sstable as compacted. I'd look for permissions
> problems.
>
> Short term there is no dire consequence, although it will keep
> re-compacting that sstable. Longer term you'll run out of disk space
> since nothing gets deleted.
how it could run of out disk space? If cassandra is able to create new
sstable, then it should be able to create empty files too.
Re: Unable to create compaction marker
Posted by Daning <da...@netseer.com>.
Thank you guys. I don't think have permission issue or out of space.
might be ulimit set to low(1024), we will change ulimit -Hn and unlimit
-Sn to see if that could solve the problem.
Daning
On 09/22/2011 01:12 AM, aaron morton wrote:
> It's in the yaml file…
>
> # directories where Cassandra should store data on disk.
> data_file_directories:
> - /var/lib/cassandra/data
>
> The permissions are normally cassandra:cassandra
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 22/09/2011, at 12:06 PM, Daning wrote:
>
>> Thanks Jonathan. where is the data directory configured? since I did not any permission problem.
>>
>> Daning
>>
>> On 09/21/2011 11:01 AM, Jonathan Ellis wrote:
>>> Means Cassandra couldn't create an empty file in the data directory
>>> designating a sstable as compacted. I'd look for permissions
>>> problems.
>>>
>>> Short term there is no dire consequence, although it will keep
>>> re-compacting that sstable. Longer term you'll run out of disk space
>>> since nothing gets deleted.
>>>
>>> On Wed, Sep 21, 2011 at 1:44 PM, Daning<da...@netseer.com> wrote:
>>>> I got this exception in system.log several times a day, do you have idea
>>>> what caused this problem and what is the consequence?
>>>>
>>>>
>>>> ERROR [CompactionExecutor:12] 2011-09-15 14:29:19,578
>>>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>>>> Thread[CompactionExecutor:12,1,main]
>>>> java.io.IOError: java.io.IOException: Unable to create compaction marker
>>>> at
>>>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:642)
>>>> at
>>>> org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:280)
>>>> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:253)
>>>> at
>>>> org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:214)
>>>> at
>>>> org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1094)
>>>> at
>>>> org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:599)
>>>> at
>>>> org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:507)
>>>> at
>>>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:142)
>>>> at
>>>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:108)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>> Source)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>> at java.lang.Thread.run(Unknown Source)
>>>> Caused by: java.io.IOException: Unable to create compaction marker
>>>> at
>>>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
>>>> ... 13 more
>>>>
>>>>
>>>> Thank you in advance.
>>>>
>>>> Daning
>>>>
>>>
Re: Unable to create compaction marker
Posted by aaron morton <aa...@thelastpickle.com>.
It's in the yaml file…
# directories where Cassandra should store data on disk.
data_file_directories:
- /var/lib/cassandra/data
The permissions are normally cassandra:cassandra
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 22/09/2011, at 12:06 PM, Daning wrote:
> Thanks Jonathan. where is the data directory configured? since I did not any permission problem.
>
> Daning
>
> On 09/21/2011 11:01 AM, Jonathan Ellis wrote:
>> Means Cassandra couldn't create an empty file in the data directory
>> designating a sstable as compacted. I'd look for permissions
>> problems.
>>
>> Short term there is no dire consequence, although it will keep
>> re-compacting that sstable. Longer term you'll run out of disk space
>> since nothing gets deleted.
>>
>> On Wed, Sep 21, 2011 at 1:44 PM, Daning<da...@netseer.com> wrote:
>>> I got this exception in system.log several times a day, do you have idea
>>> what caused this problem and what is the consequence?
>>>
>>>
>>> ERROR [CompactionExecutor:12] 2011-09-15 14:29:19,578
>>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>>> Thread[CompactionExecutor:12,1,main]
>>> java.io.IOError: java.io.IOException: Unable to create compaction marker
>>> at
>>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:642)
>>> at
>>> org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:280)
>>> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:253)
>>> at
>>> org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:214)
>>> at
>>> org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1094)
>>> at
>>> org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:599)
>>> at
>>> org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:507)
>>> at
>>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:142)
>>> at
>>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:108)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>> Source)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>> at java.lang.Thread.run(Unknown Source)
>>> Caused by: java.io.IOException: Unable to create compaction marker
>>> at
>>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
>>> ... 13 more
>>>
>>>
>>> Thank you in advance.
>>>
>>> Daning
>>>
>>
>>
>
Re: Unable to create compaction marker
Posted by Daning <da...@netseer.com>.
Thanks Jonathan. where is the data directory configured? since I did not
any permission problem.
Daning
On 09/21/2011 11:01 AM, Jonathan Ellis wrote:
> Means Cassandra couldn't create an empty file in the data directory
> designating a sstable as compacted. I'd look for permissions
> problems.
>
> Short term there is no dire consequence, although it will keep
> re-compacting that sstable. Longer term you'll run out of disk space
> since nothing gets deleted.
>
> On Wed, Sep 21, 2011 at 1:44 PM, Daning<da...@netseer.com> wrote:
>> I got this exception in system.log several times a day, do you have idea
>> what caused this problem and what is the consequence?
>>
>>
>> ERROR [CompactionExecutor:12] 2011-09-15 14:29:19,578
>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>> Thread[CompactionExecutor:12,1,main]
>> java.io.IOError: java.io.IOException: Unable to create compaction marker
>> at
>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:642)
>> at
>> org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:280)
>> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:253)
>> at
>> org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:214)
>> at
>> org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1094)
>> at
>> org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:599)
>> at
>> org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:507)
>> at
>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:142)
>> at
>> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:108)
>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> at java.lang.Thread.run(Unknown Source)
>> Caused by: java.io.IOException: Unable to create compaction marker
>> at
>> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
>> ... 13 more
>>
>>
>> Thank you in advance.
>>
>> Daning
>>
>
>
Re: Unable to create compaction marker
Posted by Jonathan Ellis <jb...@gmail.com>.
Means Cassandra couldn't create an empty file in the data directory
designating a sstable as compacted. I'd look for permissions
problems.
Short term there is no dire consequence, although it will keep
re-compacting that sstable. Longer term you'll run out of disk space
since nothing gets deleted.
On Wed, Sep 21, 2011 at 1:44 PM, Daning <da...@netseer.com> wrote:
> I got this exception in system.log several times a day, do you have idea
> what caused this problem and what is the consequence?
>
>
> ERROR [CompactionExecutor:12] 2011-09-15 14:29:19,578
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[CompactionExecutor:12,1,main]
> java.io.IOError: java.io.IOException: Unable to create compaction marker
> at
> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:642)
> at
> org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:280)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:253)
> at
> org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:214)
> at
> org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1094)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:599)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:507)
> at
> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:142)
> at
> org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:108)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Unable to create compaction marker
> at
> org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638)
> ... 13 more
>
>
> Thank you in advance.
>
> Daning
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com