You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Roberto Iannone <ro...@gmail.com> on 2011/11/28 11:54:20 UTC
help no segment in my lucene index!!!
Hi all,
after a power supply inperruption my lucene index (about 28 GB) looks like
this:
18/11/2011 20:29 2.016.961.997 _3d.fdt
18/11/2011 20:29 1.816.004 _3d.fdx
18/11/2011 20:29 89 _3d.fnm
18/11/2011 20:30 197.323.436 _3d.frq
18/11/2011 20:30 1.816.004 _3d.nrm
18/11/2011 20:30 358.016.461 _3d.prx
18/11/2011 20:30 637.604 _3d.tii
18/11/2011 20:30 48.565.519 _3d.tis
18/11/2011 20:31 454.004 _3d.tvd
18/11/2011 20:31 1.695.380.935 _3d.tvf
18/11/2011 20:31 3.632.004 _3d.tvx
18/11/2011 23:33 2.048.500.822 _6g.fdt
18/11/2011 23:33 3.032.004 _6g.fdx
18/11/2011 23:33 89 _6g.fnm
18/11/2011 23:34 221.593.644 _6g.frq
18/11/2011 23:34 3.032.004 _6g.nrm
18/11/2011 23:34 350.136.996 _6g.prx
18/11/2011 23:34 683.668 _6g.tii
18/11/2011 23:34 52.224.328 _6g.tis
18/11/2011 23:36 758.004 _6g.tvd
18/11/2011 23:36 1.758.786.158 _6g.tvf
18/11/2011 23:36 6.064.004 _6g.tvx
19/11/2011 03:29 1.966.167.843 _9j.fdt
19/11/2011 03:29 3.832.004 _9j.fdx
19/11/2011 03:28 89 _9j.fnm
19/11/2011 03:30 222.733.606 _9j.frq
19/11/2011 03:30 3.832.004 _9j.nrm
19/11/2011 03:30 324.722.843 _9j.prx
19/11/2011 03:30 715.441 _9j.tii
19/11/2011 03:30 54.488.546 _9j.tis
....
without any segment files!
I tried to fix with CheckIndex utility in lucene, but I got the following
message:
ERROR: could not read any segments file in directory
org.apache.lucene.index.IndexNotFoundException: no segments* file found in
org.a
pache.lucene.store.MMapDirectory@E:\recover_me
lockFactory=org.apache.lucene.sto
re.NativeFSLockFactory@5d36d1d7: files: [_3d.fdt, _3d.fdx, _3d.fnm,
_3d.frq, _3d
.nrm, _3d.prx, _3d.tii, _3d.tis, _3d.tvd, _3d.tvf, _3d.tvx, _6g.fdt,
_6g.fdx, _6
g.fnm, _6g.frq, _6g.nrm, _6g.prx, _6g.tii, _6g.tis, _6g.tvd, _6g.tvf,
_6g.tvx, _
9j.fdt, _9j.fdx, _9j.fnm, _9j.frq, _9j.nrm, _9j.prx, _9j.tii, _9j.tis,
_9j.tvd,
_9j.tvf, _9j.tvx, _cf.cfs, _cm.fdt, _cm.fdx, _cm.fnm, _cm.frq, _cm.nrm,
_cm.prx,
_cm.tii, _cm.tis, _cm.tvd, _cm.tvf, _cm.tvx, _ff.fdt, _ff.fdx, _ff.fnm,
_ff.frq
, _ff.nrm, _ff.prx, _ff.tii, _ff.tis, _ff.tvd, _ff.tvf, _ff.tvx, _ii.fdt,
_ii.fd
x, _ii.fnm, _ii.frq, _ii.nrm, _ii.prx, _ii.tii, _ii.tis, _ii.tvd, _ii.tvf,
_ii.t
vx, _lc.cfs, _ll.fdt, _ll.fdx, _ll.fnm, _ll.frq, _ll.nrm, _ll.prx, _ll.tii,
_ll.
tis, _ll.tvd, _ll.tvf, _ll.tvx, _lo.cfs, _lp.cfs, _lq.cfs, _lr.cfs,
_ls.cfs, _lt
.cfs, _lu.cfs, _lv.cfs, _lw.fdt, _lw.fdx, _lw.tvd, _lw.tvf, _lw.tvx,
_m.fdt, _m.
fdx, _m.fnm, _m.frq, _m.nrm, _m.prx, _m.tii, _m.tis, _m.tvd, _m.tvf, _m.tvx]
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
s.java:712)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
s.java:593)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
at
org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:327)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:995)
There's a way to recover this index ?
Cheers
Rob
Re: help no segment in my lucene index!!!
Posted by Michael McCandless <lu...@mikemccandless.com>.
On Mon, Nov 28, 2011 at 10:49 AM, Roberto Iannone
<ia...@crmpa.unisa.it> wrote:
> Hi Michael,
>
> thx for your help :)
You're welcome!
> 2011/11/28 Michael McCandless <lu...@mikemccandless.com>
>
>> Which version of Solr/Lucene were you using when you hit power loss?
>>
> I'm using Lucene 3.4.
Hmm, which OS/filesystem? Unexpected power loss (nor OS crash, JVM
crash) in 3.4.0 should not cause corrumption, as long as the IO system
properly implements fsync.
>> There was a known bug that could allow power loss to cause corruption,
>> but this was fixed in Lucene 3.4.0.
>>
>> Unfortunately, there is no easy way to recreate the segments_N file...
>> in principle it should be possible and maybe not too much work but
>> nobody has created such a tool yet, that I know of.
>
> some hints about how could I write this code by myself ?
Well, you'd need to take a listing of all files, aggregate those into
unique segment names, open a SegmentReader on each segment name, and
from that SegmentReader reconstruct what you can (numDocs, delCount,
isCompoundFile, etc.) about each SegmentInfo. Add all the resulting
SegmentInfo instances into a new SegmentInfos and write it to the
directory.
Was the index newly created in 3.4.x? If not (if you inherited
segments from earlier Lucene versions) you might also have to
reconstruct shared doc stores (stored fields, term vectors) files,
which will be trickier...
Mike
Re: help no segment in my lucene index!!!
Posted by Roberto Iannone <ia...@crmpa.unisa.it>.
Hi Michael,
thx for your help :)
2011/11/28 Michael McCandless <lu...@mikemccandless.com>
> Which version of Solr/Lucene were you using when you hit power loss?
>
> I'm using Lucene 3.4.
> There was a known bug that could allow power loss to cause corruption,
> but this was fixed in Lucene 3.4.0.
>
> Unfortunately, there is no easy way to recreate the segments_N file...
> in principle it should be possible and maybe not too much work but
> nobody has created such a tool yet, that I know of.
>
>
some hints about how could I write this code by myself ?
Cheers
Rob
Re: help no segment in my lucene index!!!
Posted by Michael McCandless <lu...@mikemccandless.com>.
Which version of Solr/Lucene were you using when you hit power loss?
There was a known bug that could allow power loss to cause corruption,
but this was fixed in Lucene 3.4.0.
Unfortunately, there is no easy way to recreate the segments_N file...
in principle it should be possible and maybe not too much work but
nobody has created such a tool yet, that I know of.
Mike McCandless
http://blog.mikemccandless.com
On Mon, Nov 28, 2011 at 5:54 AM, Roberto Iannone
<ro...@gmail.com> wrote:
> Hi all,
>
> after a power supply inperruption my lucene index (about 28 GB) looks like
> this:
>
> 18/11/2011 20:29 2.016.961.997 _3d.fdt
> 18/11/2011 20:29 1.816.004 _3d.fdx
> 18/11/2011 20:29 89 _3d.fnm
> 18/11/2011 20:30 197.323.436 _3d.frq
> 18/11/2011 20:30 1.816.004 _3d.nrm
> 18/11/2011 20:30 358.016.461 _3d.prx
> 18/11/2011 20:30 637.604 _3d.tii
> 18/11/2011 20:30 48.565.519 _3d.tis
> 18/11/2011 20:31 454.004 _3d.tvd
> 18/11/2011 20:31 1.695.380.935 _3d.tvf
> 18/11/2011 20:31 3.632.004 _3d.tvx
> 18/11/2011 23:33 2.048.500.822 _6g.fdt
> 18/11/2011 23:33 3.032.004 _6g.fdx
> 18/11/2011 23:33 89 _6g.fnm
> 18/11/2011 23:34 221.593.644 _6g.frq
> 18/11/2011 23:34 3.032.004 _6g.nrm
> 18/11/2011 23:34 350.136.996 _6g.prx
> 18/11/2011 23:34 683.668 _6g.tii
> 18/11/2011 23:34 52.224.328 _6g.tis
> 18/11/2011 23:36 758.004 _6g.tvd
> 18/11/2011 23:36 1.758.786.158 _6g.tvf
> 18/11/2011 23:36 6.064.004 _6g.tvx
> 19/11/2011 03:29 1.966.167.843 _9j.fdt
> 19/11/2011 03:29 3.832.004 _9j.fdx
> 19/11/2011 03:28 89 _9j.fnm
> 19/11/2011 03:30 222.733.606 _9j.frq
> 19/11/2011 03:30 3.832.004 _9j.nrm
> 19/11/2011 03:30 324.722.843 _9j.prx
> 19/11/2011 03:30 715.441 _9j.tii
> 19/11/2011 03:30 54.488.546 _9j.tis
> ....
>
> without any segment files!
> I tried to fix with CheckIndex utility in lucene, but I got the following
> message:
>
> ERROR: could not read any segments file in directory
> org.apache.lucene.index.IndexNotFoundException: no segments* file found in
> org.a
> pache.lucene.store.MMapDirectory@E:\recover_me
> lockFactory=org.apache.lucene.sto
> re.NativeFSLockFactory@5d36d1d7: files: [_3d.fdt, _3d.fdx, _3d.fnm,
> _3d.frq, _3d
> .nrm, _3d.prx, _3d.tii, _3d.tis, _3d.tvd, _3d.tvf, _3d.tvx, _6g.fdt,
> _6g.fdx, _6
> g.fnm, _6g.frq, _6g.nrm, _6g.prx, _6g.tii, _6g.tis, _6g.tvd, _6g.tvf,
> _6g.tvx, _
> 9j.fdt, _9j.fdx, _9j.fnm, _9j.frq, _9j.nrm, _9j.prx, _9j.tii, _9j.tis,
> _9j.tvd,
> _9j.tvf, _9j.tvx, _cf.cfs, _cm.fdt, _cm.fdx, _cm.fnm, _cm.frq, _cm.nrm,
> _cm.prx,
> _cm.tii, _cm.tis, _cm.tvd, _cm.tvf, _cm.tvx, _ff.fdt, _ff.fdx, _ff.fnm,
> _ff.frq
> , _ff.nrm, _ff.prx, _ff.tii, _ff.tis, _ff.tvd, _ff.tvf, _ff.tvx, _ii.fdt,
> _ii.fd
> x, _ii.fnm, _ii.frq, _ii.nrm, _ii.prx, _ii.tii, _ii.tis, _ii.tvd, _ii.tvf,
> _ii.t
> vx, _lc.cfs, _ll.fdt, _ll.fdx, _ll.fnm, _ll.frq, _ll.nrm, _ll.prx, _ll.tii,
> _ll.
> tis, _ll.tvd, _ll.tvf, _ll.tvx, _lo.cfs, _lp.cfs, _lq.cfs, _lr.cfs,
> _ls.cfs, _lt
> .cfs, _lu.cfs, _lv.cfs, _lw.fdt, _lw.fdx, _lw.tvd, _lw.tvf, _lw.tvx,
> _m.fdt, _m.
> fdx, _m.fnm, _m.frq, _m.nrm, _m.prx, _m.tii, _m.tis, _m.tvd, _m.tvf, _m.tvx]
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> s.java:712)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> s.java:593)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
> at
> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:327)
> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:995)
>
> There's a way to recover this index ?
>
> Cheers
>
> Rob
>