commit_log way bigger than allowed

[pwozniak <pw...@man.poznan.pl>]

Hi,

I have cluster of three Cassandra (v.2.1) machines. On one of the 
machines files with commit logs filled up all available disk space (50 GB).

I haven't change 'commitlog_total_space_in_mb', so as far as I know It 
shouldn't take more that 8GB of disc space.

I also haven't found any suspicious messages in log file and our cluster 
was not hammered by huge amount of requests lately.

This machine (cassandra process) is not able to boot up now (it crashes 
while replaying commit_log)


1. How can I find out what happened?

2. Can I just delete all commit_logs, restart machine and run repair to 
have consistent data?

3. Maybe I can delete just part of the commit_log files so Cassandra 
will be able to boot and clean (flush) all commit_log files?


Regards,

Pawel


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org

Re: commit_log way bigger than allowed

[Nitan Kainth <ni...@gmail.com>]

You can delete commitlogs and start the node. But run repair on that node to sync any data mismatch.


Regards,
Nitan
Cell: 510 449 9629

> On Jun 25, 2019, at 4:37 AM, pwozniak <pw...@man.poznan.pl> wrote:
> 
> Hi,
> 
> I have cluster of three Cassandra (v.2.1) machines. On one of the machines files with commit logs filled up all available disk space (50 GB).
> 
> I haven't change 'commitlog_total_space_in_mb', so as far as I know It shouldn't take more that 8GB of disc space.
> 
> I also haven't found any suspicious messages in log file and our cluster was not hammered by huge amount of requests lately.
> 
> This machine (cassandra process) is not able to boot up now (it crashes while replaying commit_log)
> 
> 
> 1. How can I find out what happened?
> 
> 2. Can I just delete all commit_logs, restart machine and run repair to have consistent data?
> 
> 3. Maybe I can delete just part of the commit_log files so Cassandra will be able to boot and clean (flush) all commit_log files?
> 
> 
> Regards,
> 
> Pawel
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>