You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by D Bera <db...@gmail.com> on 2007/12/09 19:08:41 UTC
cfs and temporary files (fdt, fnm etc)
Hi,
We use dotlucene-1.9 for indexing in our desktop search tool.
Recently one user complained about lots of temporary files in his
index, see the list at the end of this email. As you will see, there
are the cfs files (and correctly referenced in the segments file) but
then there are lots of fdt/fnm/fdx files and a bunch from the latest
run too. The prefixes of the f* files do not appear in the segments
file, so could it be that the process somehow crashed during indexing
but before being able to merge them to the corresponding cfs file ?
What does lucene do with the leftover f* files in such cases ? Or it
is the applications responsbility to remove the temporary files. The
user did not report any kind of crash while indexing, so is there any
other crazy possibility ? We do an Optimize() after the indexing is
over - so anything could go wrong in that ?
...
-rw-r--r-- 1 beagleindex nogroup 17611812 2007-10-23 07:41 _63qu.fdt
-rw-r--r-- 1 beagleindex nogroup 296720 2007-10-23 07:41 _63qu.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-10-23 07:41 _63qu.fnm
-rw-r--r-- 1 beagleindex nogroup 7766016 2007-10-23 07:41 _63qu.frq
-rw-r--r-- 1 beagleindex nogroup 13336576 2007-10-23 07:41 _63qu.prx
-rw-r--r-- 1 beagleindex nogroup 57344 2007-10-23 07:41 _63qu.tii
-rw-r--r-- 1 beagleindex nogroup 4046848 2007-10-23 07:41 _63qu.tis
-rw-r--r-- 1 beagleindex nogroup 15699968 2007-11-16 07:38 _68w9.fdt
-rw-r--r-- 1 beagleindex nogroup 253952 2007-11-16 07:38 _68w9.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-11-16 07:38 _68w9.fnm
-rw-r--r-- 1 beagleindex nogroup 17628575 2007-11-20 07:38 _69hn.fdt
-rw-r--r-- 1 beagleindex nogroup 297160 2007-11-20 07:38 _69hn.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-11-20 07:38 _69hn.fnm
-rw-r--r-- 1 beagleindex nogroup 7458816 2007-11-20 07:39 _69hn.frq
-rw-r--r-- 1 beagleindex nogroup 12976128 2007-11-20 07:39 _69hn.prx
-rw-r--r-- 1 beagleindex nogroup 49152 2007-11-20 07:39 _69hn.tii
-rw-r--r-- 1 beagleindex nogroup 3694592 2007-11-20 07:39 _69hn.tis
-rw-r--r-- 1 beagleindex nogroup 17910953 2007-12-07 07:38 _6zx9.fdt
-rw-r--r-- 1 beagleindex nogroup 302672 2007-12-07 07:38 _6zx9.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-12-07 07:38 _6zx9.fnm
-rw-r--r-- 1 beagleindex nogroup 6561792 2007-12-07 07:38 _6zx9.frq
-rw-r--r-- 1 beagleindex nogroup 11542528 2007-12-07 07:38 _6zx9.prx
-rw-r--r-- 1 beagleindex nogroup 32768 2007-12-07 07:38 _6zx9.tii
-rw-r--r-- 1 beagleindex nogroup 2998272 2007-12-07 07:38 _6zx9.tis
-rw-r--r-- 1 beagleindex nogroup 49329915 2007-12-08 07:38 _7048.cfs
-rw-r--r-- 1 beagleindex nogroup 4738 2007-12-09 07:41 _7048.del
-rw-r--r-- 1 beagleindex nogroup 1428220 2007-12-09 07:40 _70z3.cfs
-rw-r--r-- 1 beagleindex nogroup 1218550 2007-12-09 07:41 _71ty.cfs
-rw-r--r-- 1 beagleindex nogroup 147565 2007-12-09 07:41 _71x1.cfs
-rw-r--r-- 1 beagleindex nogroup 31054 2007-12-09 07:41 _71xc.cfs
-rw-r--r-- 1 beagleindex nogroup 17702 2007-12-09 07:41 _71xn.cfs
-rw-r--r-- 1 beagleindex nogroup 15017 2007-12-09 07:41 _71xy.cfs
-rw-r--r-- 1 beagleindex nogroup 5716 2007-12-09 07:41 _71y4.cfs
-rw-r--r-- 1 beagleindex nogroup 5259264 2007-12-09 07:41 _71y5.fdt
-rw-r--r-- 1 beagleindex nogroup 90112 2007-12-09 07:41 _71y5.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-12-09 07:41 _71y5.fnm
-rw-r--r-- 1 beagleindex nogroup 4 2007-12-09 07:41 deletable
...
Thanks,
- dBera
RE: cfs and temporary files (fdt, fnm etc)
Posted by George Aroush <ge...@aroush.net>.
I'm not aware of any bugs in 1.9 to cause such a problem -- 1.9 is actually
very solid. Still, do try the latest version like DIGY suggested, at the
same time, you should find out how the user was using Lucene to cause this
issue.
-- George
> -----Original Message-----
> From: DIGY [mailto:digydigy@gmail.com]
> Sent: Monday, December 10, 2007 12:06 PM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: cfs and temporary files (fdt, fnm etc)
>
> I would strongly recommend to upgrade to version 2.0.0.4 or
> to 2.1.0(from SVN).
> There are a lot of bugs fixed.
>
> DIGY
>
> -----Original Message-----
> From: D Bera [mailto:dbera.web@gmail.com]
> Sent: Sunday, December 09, 2007 8:09 PM
> To: lucene-net-user@incubator.apache.org
> Subject: cfs and temporary files (fdt, fnm etc)
>
> Hi,
> We use dotlucene-1.9 for indexing in our desktop search tool.
> Recently one user complained about lots of temporary files in
> his index, see the list at the end of this email. As you will
> see, there are the cfs files (and correctly referenced in the
> segments file) but then there are lots of fdt/fnm/fdx files
> and a bunch from the latest run too. The prefixes of the f*
> files do not appear in the segments file, so could it be that
> the process somehow crashed during indexing but before being
> able to merge them to the corresponding cfs file ?
> What does lucene do with the leftover f* files in such cases
> ? Or it is the applications responsbility to remove the
> temporary files. The user did not report any kind of crash
> while indexing, so is there any other crazy possibility ? We
> do an Optimize() after the indexing is over - so anything
> could go wrong in that ?
>
> ...
> -rw-r--r-- 1 beagleindex nogroup 17611812 2007-10-23 07:41 _63qu.fdt
> -rw-r--r-- 1 beagleindex nogroup 296720 2007-10-23 07:41 _63qu.fdx
> -rw-r--r-- 1 beagleindex nogroup 1426 2007-10-23 07:41 _63qu.fnm
> -rw-r--r-- 1 beagleindex nogroup 7766016 2007-10-23 07:41 _63qu.frq
> -rw-r--r-- 1 beagleindex nogroup 13336576 2007-10-23 07:41 _63qu.prx
> -rw-r--r-- 1 beagleindex nogroup 57344 2007-10-23 07:41 _63qu.tii
> -rw-r--r-- 1 beagleindex nogroup 4046848 2007-10-23 07:41 _63qu.tis
> -rw-r--r-- 1 beagleindex nogroup 15699968 2007-11-16 07:38 _68w9.fdt
> -rw-r--r-- 1 beagleindex nogroup 253952 2007-11-16 07:38 _68w9.fdx
> -rw-r--r-- 1 beagleindex nogroup 1426 2007-11-16 07:38 _68w9.fnm
> -rw-r--r-- 1 beagleindex nogroup 17628575 2007-11-20 07:38 _69hn.fdt
> -rw-r--r-- 1 beagleindex nogroup 297160 2007-11-20 07:38 _69hn.fdx
> -rw-r--r-- 1 beagleindex nogroup 1426 2007-11-20 07:38 _69hn.fnm
> -rw-r--r-- 1 beagleindex nogroup 7458816 2007-11-20 07:39 _69hn.frq
> -rw-r--r-- 1 beagleindex nogroup 12976128 2007-11-20 07:39 _69hn.prx
> -rw-r--r-- 1 beagleindex nogroup 49152 2007-11-20 07:39 _69hn.tii
> -rw-r--r-- 1 beagleindex nogroup 3694592 2007-11-20 07:39 _69hn.tis
> -rw-r--r-- 1 beagleindex nogroup 17910953 2007-12-07 07:38 _6zx9.fdt
> -rw-r--r-- 1 beagleindex nogroup 302672 2007-12-07 07:38 _6zx9.fdx
> -rw-r--r-- 1 beagleindex nogroup 1426 2007-12-07 07:38 _6zx9.fnm
> -rw-r--r-- 1 beagleindex nogroup 6561792 2007-12-07 07:38 _6zx9.frq
> -rw-r--r-- 1 beagleindex nogroup 11542528 2007-12-07 07:38 _6zx9.prx
> -rw-r--r-- 1 beagleindex nogroup 32768 2007-12-07 07:38 _6zx9.tii
> -rw-r--r-- 1 beagleindex nogroup 2998272 2007-12-07 07:38 _6zx9.tis
> -rw-r--r-- 1 beagleindex nogroup 49329915 2007-12-08 07:38 _7048.cfs
> -rw-r--r-- 1 beagleindex nogroup 4738 2007-12-09 07:41 _7048.del
> -rw-r--r-- 1 beagleindex nogroup 1428220 2007-12-09 07:40 _70z3.cfs
> -rw-r--r-- 1 beagleindex nogroup 1218550 2007-12-09 07:41 _71ty.cfs
> -rw-r--r-- 1 beagleindex nogroup 147565 2007-12-09 07:41 _71x1.cfs
> -rw-r--r-- 1 beagleindex nogroup 31054 2007-12-09 07:41 _71xc.cfs
> -rw-r--r-- 1 beagleindex nogroup 17702 2007-12-09 07:41 _71xn.cfs
> -rw-r--r-- 1 beagleindex nogroup 15017 2007-12-09 07:41 _71xy.cfs
> -rw-r--r-- 1 beagleindex nogroup 5716 2007-12-09 07:41 _71y4.cfs
> -rw-r--r-- 1 beagleindex nogroup 5259264 2007-12-09 07:41 _71y5.fdt
> -rw-r--r-- 1 beagleindex nogroup 90112 2007-12-09 07:41 _71y5.fdx
> -rw-r--r-- 1 beagleindex nogroup 1426 2007-12-09 07:41 _71y5.fnm
> -rw-r--r-- 1 beagleindex nogroup 4 2007-12-09 07:41 deletable
> ...
>
> Thanks,
>
> - dBera
>
RE: cfs and temporary files (fdt, fnm etc)
Posted by DIGY <di...@gmail.com>.
I would strongly recommend to upgrade to version 2.0.0.4 or to 2.1.0(from
SVN).
There are a lot of bugs fixed.
DIGY
-----Original Message-----
From: D Bera [mailto:dbera.web@gmail.com]
Sent: Sunday, December 09, 2007 8:09 PM
To: lucene-net-user@incubator.apache.org
Subject: cfs and temporary files (fdt, fnm etc)
Hi,
We use dotlucene-1.9 for indexing in our desktop search tool.
Recently one user complained about lots of temporary files in his
index, see the list at the end of this email. As you will see, there
are the cfs files (and correctly referenced in the segments file) but
then there are lots of fdt/fnm/fdx files and a bunch from the latest
run too. The prefixes of the f* files do not appear in the segments
file, so could it be that the process somehow crashed during indexing
but before being able to merge them to the corresponding cfs file ?
What does lucene do with the leftover f* files in such cases ? Or it
is the applications responsbility to remove the temporary files. The
user did not report any kind of crash while indexing, so is there any
other crazy possibility ? We do an Optimize() after the indexing is
over - so anything could go wrong in that ?
...
-rw-r--r-- 1 beagleindex nogroup 17611812 2007-10-23 07:41 _63qu.fdt
-rw-r--r-- 1 beagleindex nogroup 296720 2007-10-23 07:41 _63qu.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-10-23 07:41 _63qu.fnm
-rw-r--r-- 1 beagleindex nogroup 7766016 2007-10-23 07:41 _63qu.frq
-rw-r--r-- 1 beagleindex nogroup 13336576 2007-10-23 07:41 _63qu.prx
-rw-r--r-- 1 beagleindex nogroup 57344 2007-10-23 07:41 _63qu.tii
-rw-r--r-- 1 beagleindex nogroup 4046848 2007-10-23 07:41 _63qu.tis
-rw-r--r-- 1 beagleindex nogroup 15699968 2007-11-16 07:38 _68w9.fdt
-rw-r--r-- 1 beagleindex nogroup 253952 2007-11-16 07:38 _68w9.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-11-16 07:38 _68w9.fnm
-rw-r--r-- 1 beagleindex nogroup 17628575 2007-11-20 07:38 _69hn.fdt
-rw-r--r-- 1 beagleindex nogroup 297160 2007-11-20 07:38 _69hn.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-11-20 07:38 _69hn.fnm
-rw-r--r-- 1 beagleindex nogroup 7458816 2007-11-20 07:39 _69hn.frq
-rw-r--r-- 1 beagleindex nogroup 12976128 2007-11-20 07:39 _69hn.prx
-rw-r--r-- 1 beagleindex nogroup 49152 2007-11-20 07:39 _69hn.tii
-rw-r--r-- 1 beagleindex nogroup 3694592 2007-11-20 07:39 _69hn.tis
-rw-r--r-- 1 beagleindex nogroup 17910953 2007-12-07 07:38 _6zx9.fdt
-rw-r--r-- 1 beagleindex nogroup 302672 2007-12-07 07:38 _6zx9.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-12-07 07:38 _6zx9.fnm
-rw-r--r-- 1 beagleindex nogroup 6561792 2007-12-07 07:38 _6zx9.frq
-rw-r--r-- 1 beagleindex nogroup 11542528 2007-12-07 07:38 _6zx9.prx
-rw-r--r-- 1 beagleindex nogroup 32768 2007-12-07 07:38 _6zx9.tii
-rw-r--r-- 1 beagleindex nogroup 2998272 2007-12-07 07:38 _6zx9.tis
-rw-r--r-- 1 beagleindex nogroup 49329915 2007-12-08 07:38 _7048.cfs
-rw-r--r-- 1 beagleindex nogroup 4738 2007-12-09 07:41 _7048.del
-rw-r--r-- 1 beagleindex nogroup 1428220 2007-12-09 07:40 _70z3.cfs
-rw-r--r-- 1 beagleindex nogroup 1218550 2007-12-09 07:41 _71ty.cfs
-rw-r--r-- 1 beagleindex nogroup 147565 2007-12-09 07:41 _71x1.cfs
-rw-r--r-- 1 beagleindex nogroup 31054 2007-12-09 07:41 _71xc.cfs
-rw-r--r-- 1 beagleindex nogroup 17702 2007-12-09 07:41 _71xn.cfs
-rw-r--r-- 1 beagleindex nogroup 15017 2007-12-09 07:41 _71xy.cfs
-rw-r--r-- 1 beagleindex nogroup 5716 2007-12-09 07:41 _71y4.cfs
-rw-r--r-- 1 beagleindex nogroup 5259264 2007-12-09 07:41 _71y5.fdt
-rw-r--r-- 1 beagleindex nogroup 90112 2007-12-09 07:41 _71y5.fdx
-rw-r--r-- 1 beagleindex nogroup 1426 2007-12-09 07:41 _71y5.fnm
-rw-r--r-- 1 beagleindex nogroup 4 2007-12-09 07:41 deletable
...
Thanks,
- dBera