files left open?

[Paulo Magalhaes <pa...@gmail.com>]

Hi all,

I was loading a big (60 million docs) csv in solr 4 when something odd
happened.
I got a solr error in the log saying that it could not write the file.
du -s indicated I had used 30Gb of a 50Gb available but df -k  indicated
that the disk was I00% used.
ds and df giving different results could be an indication that there are
file descriptors left open.
After a solr bounce, df -k came down and agreed with du.
has anyone seen anything like that ?

Thanks,
Paulo.

environment;
Linux 2.6.18-238.19.1.el5.centos.
plus #1 SMP Mon Jul 18 10:05:09 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
Java(TM) SE Runtime Environment (build 1.6.0_17-b04) Java HotSpot(TM)
64-Bit Server VM (build 14.3-b01, mixed mode)
apache-solr-4.0-2012-02-10_09-58-50
solr config is the one in the distribution package. i had my own schema.

Re: files left open?

[Yonik Seeley <yo...@lucidimagination.com>]

On Thu, Feb 16, 2012 at 5:56 PM, Paulo Magalhaes
<pa...@gmail.com> wrote:
> I was loading a big (60 million docs) csv in solr 4 when something odd
> happened.
> I got a solr error in the log saying that it could not write the file.
> du -s indicated I had used 30Gb of a 50Gb available but df -k  indicated
> that the disk was I00% used.

You probably hit a big segment merge, which does require more disk
space temporarily.
The difference between du and df probably just indicates how they
internally work (du may just look at file sizes, and non-closed files
can register as smaller or 0 than the amount of disk space they
actually take up).

-Yonik
lucidimagination.com