You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jeff Ferland <jb...@tubularlabs.com> on 2015/06/11 01:15:58 UTC
Files not removed after compaction
Compaction finished yesterday, but I still have this going on:
Space used (live), bytes: 878681716067
Space used (total), bytes: 2227857083852
jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ sudo lsof *-Data.db
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 4473 cassandra 446r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db
java 4473 cassandra 448r REG 0,26 62040962 37431 trends-trends-jb-144731-Data.db
java 4473 cassandra 449r REG 0,26 829935047545 21150 trends-trends-jb-143581-Data.db
java 4473 cassandra 452r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db
java 4473 cassandra 454r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db
java 4473 cassandra 462r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db
java 4473 cassandra 463r REG 0,26 36158226 39629 trends-trends-jb-144889-Data.db
java 4473 cassandra 468r REG 0,26 105693505 39447 trends-trends-jb-144881-Data.db
java 4473 cassandra 530r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db
java 4473 cassandra 535r REG 0,26 105693505 39447 trends-trends-jb-144881-Data.db
java 4473 cassandra 542r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db
java 4473 cassandra 553u REG 0,26 6431729821 39556 trends-trends-tmp-jb-144884-Data.db
jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ ls *-Data.db
trends-trends-jb-142631-Data.db trends-trends-jb-143562-Data.db trends-trends-jb-143581-Data.db trends-trends-jb-144731-Data.db trends-trends-jb-144883-Data.db
trends-trends-jb-142633-Data.db trends-trends-jb-143563-Data.db trends-trends-jb-144530-Data.db trends-trends-jb-144864-Data.db trends-trends-jb-144889-Data.db
trends-trends-jb-143026-Data.db trends-trends-jb-143564-Data.db trends-trends-jb-144551-Data.db trends-trends-jb-144881-Data.db trends-trends-tmp-jb-144884-Data.db
trends-trends-jb-143533-Data.db trends-trends-jb-143578-Data.db trends-trends-jb-144552-Data.db trends-trends-jb-144882-Data.db
jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ cd -
/mnt/cassandra/data/trends/trends
jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ sudo lsof *
jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ ls *-Data.db
trends-trends-jb-124502-Data.db trends-trends-jb-141113-Data.db trends-trends-jb-141377-Data.db trends-trends-jb-141846-Data.db trends-trends-jb-144890-Data.db
trends-trends-jb-125457-Data.db trends-trends-jb-141123-Data.db trends-trends-jb-141391-Data.db trends-trends-jb-141871-Data.db trends-trends-jb-41121-Data.db
trends-trends-jb-130016-Data.db trends-trends-jb-141137-Data.db trends-trends-jb-141538-Data.db trends-trends-jb-141883-Data.db trends-trends.trends_date_idx-jb-2100-Data.db
trends-trends-jb-139563-Data.db trends-trends-jb-141358-Data.db trends-trends-jb-141806-Data.db trends-trends-jb-142033-Data.db
trends-trends-jb-141102-Data.db trends-trends-jb-141363-Data.db trends-trends-jb-141829-Data.db trends-trends-jb-144553-Data.db
Essentially, everything was brought down in a major compaction but Cassandra won’t let go of the files and I can’t figure out as to why they’re not being cleaned up even when they’re not open. I’d really like to get those files cleared out safely and rsync away from the EBS volume I used to survive a compaction.
While I’m at it, is there any way to anti-compact a table by year? Since I’ve switched this recently to date tiered compaction, breaking up that big one would probably be helpful.
-Jeff
Re: Files not removed after compaction
Posted by Jeff Ferland <jb...@tubularlabs.com>.
Cassandra 2.0.12-200 / DSE 4.6.1 https://issues.apache.org/jira/browse/CASSANDRA-9577 <https://issues.apache.org/jira/browse/CASSANDRA-9577>
-Jeff
> On Jun 10, 2015, at 4:36 PM, Robert Coli <rc...@eventbrite.com> wrote:
>
> On Wed, Jun 10, 2015 at 4:15 PM, Jeff Ferland <jbf@tubularlabs.com <ma...@tubularlabs.com>> wrote:
> Compaction finished yesterday, but I still have this going on:
>
> 1) What version?
> 2) Probably file a JIRA, esp. if you can repro on another node?
> 3) if you do 2, reply to thread so we can track!
>
> =Rob
Re: Files not removed after compaction
Posted by Robert Coli <rc...@eventbrite.com>.
On Wed, Jun 10, 2015 at 4:15 PM, Jeff Ferland <jb...@tubularlabs.com> wrote:
> Compaction finished yesterday, but I still have this going on:
>
1) What version?
2) Probably file a JIRA, esp. if you can repro on another node?
3) if you do 2, reply to thread so we can track!
=Rob
Re: Files not removed after compaction
Posted by Spencer Brown <li...@gmail.com>.
You can't delete by year unless tables are organized by year. For deleting
tables, cassandra keeeps a copy of the file history which you can delete.
Also, you could delete all the extra rows using CQL.
On Wed, Jun 10, 2015 at 7:15 PM, Jeff Ferland <jb...@tubularlabs.com> wrote:
> Compaction finished yesterday, but I still have this going on:
>
> Space used (live), bytes: 878681716067
> Space used (total), bytes: 2227857083852
>
> jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ sudo lsof *-Data.db
> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> java 4473 cassandra 446r REG 0,26 17582559172 39241
> trends-trends-jb-144864-Data.db
> java 4473 cassandra 448r REG 0,26 62040962 37431
> trends-trends-jb-144731-Data.db
> java 4473 cassandra 449r REG 0,26 829935047545 21150
> trends-trends-jb-143581-Data.db
> java 4473 cassandra 452r REG 0,26 8980406 39503
> trends-trends-jb-144882-Data.db
> java 4473 cassandra 454r REG 0,26 8980406 39503
> trends-trends-jb-144882-Data.db
> java 4473 cassandra 462r REG 0,26 9487703 39542
> trends-trends-jb-144883-Data.db
> java 4473 cassandra 463r REG 0,26 36158226 39629
> trends-trends-jb-144889-Data.db
> java 4473 cassandra 468r REG 0,26 105693505 39447
> trends-trends-jb-144881-Data.db
> java 4473 cassandra 530r REG 0,26 17582559172 39241
> trends-trends-jb-144864-Data.db
> java 4473 cassandra 535r REG 0,26 105693505 39447
> trends-trends-jb-144881-Data.db
> java 4473 cassandra 542r REG 0,26 9487703 39542
> trends-trends-jb-144883-Data.db
> java 4473 cassandra 553u REG 0,26 6431729821 39556
> trends-trends-tmp-jb-144884-Data.db
> jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ ls *-Data.db
> trends-trends-jb-142631-Data.db trends-trends-jb-143562-Data.db
> trends-trends-jb-143581-Data.db trends-trends-jb-144731-Data.db
> trends-trends-jb-144883-Data.db
> trends-trends-jb-142633-Data.db trends-trends-jb-143563-Data.db
> trends-trends-jb-144530-Data.db trends-trends-jb-144864-Data.db
> trends-trends-jb-144889-Data.db
> trends-trends-jb-143026-Data.db trends-trends-jb-143564-Data.db
> trends-trends-jb-144551-Data.db trends-trends-jb-144881-Data.db
> trends-trends-tmp-jb-144884-Data.db
> trends-trends-jb-143533-Data.db trends-trends-jb-143578-Data.db
> trends-trends-jb-144552-Data.db trends-trends-jb-144882-Data.db
> jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ cd -
> /mnt/cassandra/data/trends/trends
> jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ sudo lsof *
> jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ ls *-Data.db
> trends-trends-jb-124502-Data.db trends-trends-jb-141113-Data.db
> trends-trends-jb-141377-Data.db trends-trends-jb-141846-Data.db
> trends-trends-jb-144890-Data.db
> trends-trends-jb-125457-Data.db trends-trends-jb-141123-Data.db
> trends-trends-jb-141391-Data.db trends-trends-jb-141871-Data.db
> trends-trends-jb-41121-Data.db
> trends-trends-jb-130016-Data.db trends-trends-jb-141137-Data.db
> trends-trends-jb-141538-Data.db trends-trends-jb-141883-Data.db
> trends-trends.trends_date_idx-jb-2100-Data.db
> trends-trends-jb-139563-Data.db trends-trends-jb-141358-Data.db
> trends-trends-jb-141806-Data.db trends-trends-jb-142033-Data.db
> trends-trends-jb-141102-Data.db trends-trends-jb-141363-Data.db
> trends-trends-jb-141829-Data.db trends-trends-jb-144553-Data.db
>
> Essentially, everything was brought down in a major compaction but
> Cassandra won’t let go of the files and I can’t figure out as to why
> they’re not being cleaned up even when they’re not open. I’d really like to
> get those files cleared out safely and rsync away from the EBS volume I
> used to survive a compaction.
>
> While I’m at it, is there any way to anti-compact a table by year? Since
> I’ve switched this recently to date tiered compaction, breaking up that big
> one would probably be helpful.
>
> -Jeff
>