Really old files in the data directory

[Maxim Potekhin <po...@bnl.gov>]

Hello,

a varied the GC grace a few times over the period of my cluster's 
lifetime, but I never went above
10 days. I did compactions, repairs etc. Now, I see that some files in 
the data directories of the nodes
that were there from day one carry timestamps back from July.  There are 
files containing secondary
indexes. But I have deleted a large portion of the data, one would 
expect that these files must have
been rebuilt one or many times. What's happening?

I run 0.8.6.

Thanks

Maxim

Re: Really old files in the data directory

[Edward Capriolo <ed...@gmail.com>]

Aha. I just tested that. Good point. It just seems that way because if I do
not clear my snapshots in production I run into disk space issues rather
quickly. That is what happens when you assume.

On Fri, Dec 9, 2011 at 4:09 PM, Brandon Williams <dr...@gmail.com> wrote:

> On Fri, Dec 9, 2011 at 1:57 PM, Edward Capriolo <ed...@gmail.com>
> wrote:
> > Are you taking snapshots? If the file is referenced in a snapshot it will
> > not delete until it is also not part of any snapshot.
>
> That isn't true.  The file will be removed from the data dir, but
> still exist in the snapshot dir.
>
> -Brandon
>

Re: Really old files in the data directory

[Brandon Williams <dr...@gmail.com>]

On Fri, Dec 9, 2011 at 1:57 PM, Edward Capriolo <ed...@gmail.com> wrote:
> Are you taking snapshots? If the file is referenced in a snapshot it will
> not delete until it is also not part of any snapshot.

That isn't true.  The file will be removed from the data dir, but
still exist in the snapshot dir.

-Brandon

Re: Really old files in the data directory

[Edward Capriolo <ed...@gmail.com>]

Are you taking snapshots? If the file is referenced in a snapshot it will
not delete until it is also not part of any snapshot.

On Fri, Dec 9, 2011 at 1:45 PM, Maxim Potekhin <po...@bnl.gov> wrote:

> Hello,
>
> a varied the GC grace a few times over the period of my cluster's
> lifetime, but I never went above
> 10 days. I did compactions, repairs etc. Now, I see that some files in the
> data directories of the nodes
> that were there from day one carry timestamps back from July.  There are
> files containing secondary
> indexes. But I have deleted a large portion of the data, one would expect
> that these files must have
> been rebuilt one or many times. What's happening?
>
> I run 0.8.6.
>
> Thanks
>
> Maxim
>
>