You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Henrik Schröder <sk...@gmail.com> on 2012/03/01 13:01:19 UTC
Huge amount of empty files in data directory.
Hi,
We're running Cassandra 1.0.6 on Windows, and noticed that the amount of
files in the datadirectory just keeps growing. We have about 60GB of data
per node, we do a major compaction about once a week, but after compaction
there's a lot of 0-byte temp files and old files that are kept for some
reason. After 50 days of uptime there was around 50000 files in each
datadirectory, but when we restarted a server it deleted all the
unnecessary files and it shrunk down to about 200 files.
We're running without compression, and with the regular compaction
strategy, not leveldb. I don't remember seeing this behaviour in older
versions of Cassandra, shouldn't it delete temp files while running? Is it
possible to force it to delete temp files while running? Is this fixed in a
later version? Or do we have to periodically restart servers to clean up
the datadirectories?
/Henrik Schröder
Re: Huge amount of empty files in data directory.
Posted by Viktor Jevdokimov <vj...@gmail.com>.
After running Cassandra for 2 years in production on Windows servers,
starting from 0.7 beta2 up to 1.0.7 we have moved to Linux and forgot all
the hell we had on Windows. Having JNA, off-heap row cache and normally
working MMAP on Linux you're getting a lot better performance and stability
comparing to Windows, and less maintenance.
2012/3/1 Henrik Schröder <sk...@gmail.com>
> Great, thanks!
>
>
> /Henrik
>
>
> On Thu, Mar 1, 2012 at 13:08, Sylvain Lebresne <sy...@datastax.com>wrote:
>
>> It's a bug, namely: https://issues.apache.org/jira/browse/CASSANDRA-3616
>> You'd want to upgrade.
>>
>> --
>> Sylvain
>>
>> On Thu, Mar 1, 2012 at 1:01 PM, Henrik Schröder <sk...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > We're running Cassandra 1.0.6 on Windows, and noticed that the amount of
>> > files in the datadirectory just keeps growing. We have about 60GB of
>> data
>> > per node, we do a major compaction about once a week, but after
>> compaction
>> > there's a lot of 0-byte temp files and old files that are kept for some
>> > reason. After 50 days of uptime there was around 50000 files in each
>> > datadirectory, but when we restarted a server it deleted all the
>> unnecessary
>> > files and it shrunk down to about 200 files.
>> >
>> > We're running without compression, and with the regular compaction
>> strategy,
>> > not leveldb. I don't remember seeing this behaviour in older versions of
>> > Cassandra, shouldn't it delete temp files while running? Is it possible
>> to
>> > force it to delete temp files while running? Is this fixed in a later
>> > version? Or do we have to periodically restart servers to clean up the
>> > datadirectories?
>> >
>> >
>> > /Henrik Schröder
>>
>
>
Re: Huge amount of empty files in data directory.
Posted by Henrik Schröder <sk...@gmail.com>.
Great, thanks!
/Henrik
On Thu, Mar 1, 2012 at 13:08, Sylvain Lebresne <sy...@datastax.com> wrote:
> It's a bug, namely: https://issues.apache.org/jira/browse/CASSANDRA-3616
> You'd want to upgrade.
>
> --
> Sylvain
>
> On Thu, Mar 1, 2012 at 1:01 PM, Henrik Schröder <sk...@gmail.com> wrote:
> > Hi,
> >
> > We're running Cassandra 1.0.6 on Windows, and noticed that the amount of
> > files in the datadirectory just keeps growing. We have about 60GB of data
> > per node, we do a major compaction about once a week, but after
> compaction
> > there's a lot of 0-byte temp files and old files that are kept for some
> > reason. After 50 days of uptime there was around 50000 files in each
> > datadirectory, but when we restarted a server it deleted all the
> unnecessary
> > files and it shrunk down to about 200 files.
> >
> > We're running without compression, and with the regular compaction
> strategy,
> > not leveldb. I don't remember seeing this behaviour in older versions of
> > Cassandra, shouldn't it delete temp files while running? Is it possible
> to
> > force it to delete temp files while running? Is this fixed in a later
> > version? Or do we have to periodically restart servers to clean up the
> > datadirectories?
> >
> >
> > /Henrik Schröder
>
Re: Huge amount of empty files in data directory.
Posted by Sylvain Lebresne <sy...@datastax.com>.
It's a bug, namely: https://issues.apache.org/jira/browse/CASSANDRA-3616
You'd want to upgrade.
--
Sylvain
On Thu, Mar 1, 2012 at 1:01 PM, Henrik Schröder <sk...@gmail.com> wrote:
> Hi,
>
> We're running Cassandra 1.0.6 on Windows, and noticed that the amount of
> files in the datadirectory just keeps growing. We have about 60GB of data
> per node, we do a major compaction about once a week, but after compaction
> there's a lot of 0-byte temp files and old files that are kept for some
> reason. After 50 days of uptime there was around 50000 files in each
> datadirectory, but when we restarted a server it deleted all the unnecessary
> files and it shrunk down to about 200 files.
>
> We're running without compression, and with the regular compaction strategy,
> not leveldb. I don't remember seeing this behaviour in older versions of
> Cassandra, shouldn't it delete temp files while running? Is it possible to
> force it to delete temp files while running? Is this fixed in a later
> version? Or do we have to periodically restart servers to clean up the
> datadirectories?
>
>
> /Henrik Schröder