You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Ian Lea <ia...@gmail.com> on 2014/09/10 13:01:27 UTC
4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo
unless codec is Lucene3x (got: Lucene40)
Hi
On running a quick test after a handful of minor code changes to deal
with 4.10 deprecations, a program that updates an existing index
failed with
Exception in thread "main" java.lang.IllegalStateException: cannot
write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
and along the way did something to the index to make it unusable.
Digging a bit deeper and working on a different old test index that
was lying around, and taking a backup first this time, this is
reproducible.
The working index:
total 1036
-rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
-rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
-rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
-rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
-rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
-rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
-rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
-rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
-rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
and output from 4.10 CheckIndex
Opening index @ index/
Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
1 of 1: name=_0 docCount=15730
version=4.0.0.2
codec=Lucene40
compound=false
numFiles=7
size (MB)=0.987
diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
Corporation}
no deletions
test: open reader.........OK
test: check integrity.....OK
test: check live docs.....OK
test: fields..............OK [13 fields]
test: field norms.........OK [0 fields]
test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
pairs; 139382 tokens]
test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]
test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
No problems were detected with this index.
Now run this little program
public static void main(final String[] _args) throws Exception {
File index = new File(_args[0]);
IndexWriterConfig iwcfg = new IndexWriterConfig(Version.LUCENE_4_10_0,
new StandardAnalyzer());
iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
IndexWriter iw = new IndexWriter(d, iwcfg);
Document doc1 = new Document();
doc1.add(new StringField("type", "test", Field.Store.NO));
iw.addDocument(doc1);
iw.close();
}
and it fails with
Exception in thread "main" java.lang.IllegalStateException: cannot
write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:1017)
at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3062)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169)
at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
at t.main(t.java:25)
and when run CheckIndex again get
Opening index @ index/
ERROR: could not read any segments file in directory
java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:176)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:334)
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196)
at org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:52)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
which is true
total 1032
-rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
-rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
-rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
-rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
-rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
-rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
-rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
-rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
I don't recall the origins of this index but it may well have been
created in the distant past and been upgraded, explicitly or
automatically, along the way. Although evidently not for a while.
Running the same test with lucene 4.9.0 and minimal mods to the
program runs to successful completion. Here's the CheckIndex output:
Opening index @ index-4.9.updated/
Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
1 of 2: name=_0 docCount=15730
codec=Lucene40
compound=false
numFiles=7
size (MB)=0.987
diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
Corporation}
no deletions
test: open reader.........OK
test: check integrity.....OK
test: check live docs.....OK
test: fields..............OK [13 fields]
test: field norms.........OK [0 fields]
test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
pairs; 139382 tokens]
test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]
test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
2 of 2: name=_1 docCount=1
codec=Lucene49
compound=true
numFiles=3
size (MB)=0.001
diagnostics = {timestamp=1410281698360, os=Linux,
os.version=3.1.0-1.2-desktop, source=flush, lucene.version=4.9.0
1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
java.version=1.7.0_10, java.vendor=Oracle Corporation}
no deletions
test: open reader.........OK
test: check integrity.....OK
test: check live docs.....OK
test: fields..............OK [1 fields]
test: field norms.........OK [0 fields]
test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
test: stored fields.......OK [0 total field count; avg 0 fields per doc]
test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]
test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
No problems were detected with this index.
--
Ian.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: 4.10.0: java.lang.IllegalStateException: cannot write 3x
SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Ian Lea <ia...@gmail.com>.
Sent to your personal email address.
--
Ian.
On Wed, Sep 10, 2014 at 12:36 PM, Robert Muir <rc...@gmail.com> wrote:
> Ian, this looks terrible, thanks for reporting this. Is there any
> possible way I could have a copy of that "working" index to make it
> easier to reproduce?
>
> On Wed, Sep 10, 2014 at 7:01 AM, Ian Lea <ia...@gmail.com> wrote:
>> Hi
>>
>>
>> On running a quick test after a handful of minor code changes to deal
>> with 4.10 deprecations, a program that updates an existing index
>> failed with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot
>> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
>> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>>
>> and along the way did something to the index to make it unusable.
>>
>> Digging a bit deeper and working on a different old test index that
>> was lying around, and taking a backup first this time, this is
>> reproducible.
>>
>> The working index:
>>
>> total 1036
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>> and output from 4.10 CheckIndex
>>
>> Opening index @ index/
>>
>> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
>> 1 of 1: name=_0 docCount=15730
>> version=4.0.0.2
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
>> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
>> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
>> Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
>> pairs; 139382 tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>> Now run this little program
>>
>> public static void main(final String[] _args) throws Exception {
>> File index = new File(_args[0]);
>> IndexWriterConfig iwcfg = new IndexWriterConfig(Version.LUCENE_4_10_0,
>> new StandardAnalyzer());
>> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
>> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
>> IndexWriter iw = new IndexWriter(d, iwcfg);
>> Document doc1 = new Document();
>> doc1.add(new StringField("type", "test", Field.Store.NO));
>> iw.addDocument(doc1);
>> iw.close();
>> }
>>
>> and it fails with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot
>> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
>> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
>> at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:1017)
>> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
>> at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3062)
>> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169)
>> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
>> at t.main(t.java:25)
>>
>> and when run CheckIndex again get
>>
>>
>> Opening index @ index/
>>
>> ERROR: could not read any segments file in directory
>> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si
>> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>> at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:176)
>> at java.nio.channels.FileChannel.open(FileChannel.java:287)
>> at java.nio.channels.FileChannel.open(FileChannel.java:334)
>> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196)
>> at org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:52)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
>> at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
>> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913)
>> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
>> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
>> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>>
>> which is true
>>
>> total 1032
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>>
>> I don't recall the origins of this index but it may well have been
>> created in the distant past and been upgraded, explicitly or
>> automatically, along the way. Although evidently not for a while.
>>
>>
>> Running the same test with lucene 4.9.0 and minimal mods to the
>> program runs to successful completion. Here's the CheckIndex output:
>>
>>
>> Opening index @ index-4.9.updated/
>>
>> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
>> 1 of 2: name=_0 docCount=15730
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
>> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
>> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
>> Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
>> pairs; 139382 tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> 2 of 2: name=_1 docCount=1
>> codec=Lucene49
>> compound=true
>> numFiles=3
>> size (MB)=0.001
>> diagnostics = {timestamp=1410281698360, os=Linux,
>> os.version=3.1.0-1.2-desktop, source=flush, lucene.version=4.9.0
>> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [1 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
>> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>>
>>
>> --
>> Ian.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: 4.10.0: java.lang.IllegalStateException: cannot write 3x
SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Robert Muir <rc...@gmail.com>.
Ian, this looks terrible, thanks for reporting this. Is there any
possible way I could have a copy of that "working" index to make it
easier to reproduce?
On Wed, Sep 10, 2014 at 7:01 AM, Ian Lea <ia...@gmail.com> wrote:
> Hi
>
>
> On running a quick test after a handful of minor code changes to deal
> with 4.10 deprecations, a program that updates an existing index
> failed with
>
> Exception in thread "main" java.lang.IllegalStateException: cannot
> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>
> and along the way did something to the index to make it unusable.
>
> Digging a bit deeper and working on a different old test index that
> was lying around, and taking a backup first this time, this is
> reproducible.
>
> The working index:
>
> total 1036
> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>
> and output from 4.10 CheckIndex
>
> Opening index @ index/
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
> version=4.0.0.2
> codec=Lucene40
> compound=false
> numFiles=7
> size (MB)=0.987
> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
> Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [13 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> pairs; 139382 tokens]
> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0
> term/freq vector fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> No problems were detected with this index.
>
>
> Now run this little program
>
> public static void main(final String[] _args) throws Exception {
> File index = new File(_args[0]);
> IndexWriterConfig iwcfg = new IndexWriterConfig(Version.LUCENE_4_10_0,
> new StandardAnalyzer());
> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
> IndexWriter iw = new IndexWriter(d, iwcfg);
> Document doc1 = new Document();
> doc1.add(new StringField("type", "test", Field.Store.NO));
> iw.addDocument(doc1);
> iw.close();
> }
>
> and it fails with
>
> Exception in thread "main" java.lang.IllegalStateException: cannot
> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
> at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:1017)
> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
> at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3062)
> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169)
> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
> at t.main(t.java:25)
>
> and when run CheckIndex again get
>
>
> Opening index @ index/
>
> ERROR: could not read any segments file in directory
> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si
> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:176)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:334)
> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196)
> at org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:52)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
> at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>
> which is true
>
> total 1032
> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>
>
> I don't recall the origins of this index but it may well have been
> created in the distant past and been upgraded, explicitly or
> automatically, along the way. Although evidently not for a while.
>
>
> Running the same test with lucene 4.9.0 and minimal mods to the
> program runs to successful completion. Here's the CheckIndex output:
>
>
> Opening index @ index-4.9.updated/
>
> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
> 1 of 2: name=_0 docCount=15730
> codec=Lucene40
> compound=false
> numFiles=7
> size (MB)=0.987
> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
> Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [13 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> pairs; 139382 tokens]
> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0
> term/freq vector fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> 2 of 2: name=_1 docCount=1
> codec=Lucene49
> compound=true
> numFiles=3
> size (MB)=0.001
> diagnostics = {timestamp=1410281698360, os=Linux,
> os.version=3.1.0-1.2-desktop, source=flush, lucene.version=4.9.0
> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
> java.version=1.7.0_10, java.vendor=Oracle Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [1 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0
> term/freq vector fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> No problems were detected with this index.
>
>
>
>
> --
> Ian.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
we looked into earlier releases:
The index version number of 4.0-ALPHA was "4.0"
The index version number of 4.0-BETA was "4.0.0.1"
The index version number of 4.0 final was "4.0.0.2"
Ian's index is there fore a real official 4.0 index.
Unfortunately the version comparison logic in Lucene 4.10 is wrong, as it has a special case for ALPHA and BETA indexes, which does not fit reality. Also, the constants are wrong:
/**
* Match settings and bugs in Lucene's 4.0.0-ALPHA release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0_ALPHA = new Version(4, 0, 0, 1);
/**
* Match settings and bugs in Lucene's 4.0.0-BETA release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0_BETA = new Version(4, 0, 0, 2);
/**
* Match settings and bugs in Lucene's 4.0.0 release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0 = new Version(4, 0, 0);
Because of this and the special case, 4.0.0.2 orders before "4.0.0" (see encodedVersionNumber). This causes IndexReader/IndexWriter to think it was created in 3.x.
TestBackwards compatibility did not find that bug, because the backwards index in the tests directory was created with the Alpha version :(
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Wednesday, September 10, 2014 1:42 PM
> To: java-user@lucene.apache.org
> Subject: RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> SegmentInfo unless codec is Lucene3x (got: Lucene40)
>
> Hi Ian,
>
> this index was created with the BETA version of Lucene 4.0:
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
>
> "4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not a
> supported version and may not open correctly. In Lucene 4.10 we changed
> version handling and parsing version numbers a bit, so this may be the cause
> for the error.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Ian Lea [mailto:ian.lea@gmail.com]
> > Sent: Wednesday, September 10, 2014 1:01 PM
> > To: java-user@lucene.apache.org
> > Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> > SegmentInfo unless codec is Lucene3x (got: Lucene40)
> >
> > Hi
> >
> >
> > On running a quick test after a handful of minor code changes to deal
> > with
> > 4.10 deprecations, a program that updates an existing index failed
> > with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > )
> >
> > and along the way did something to the index to make it unusable.
> >
> > Digging a bit deeper and working on a different old test index that
> > was lying around, and taking a backup first this time, this is reproducible.
> >
> > The working index:
> >
> > total 1036
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> > and output from 4.10 CheckIndex
> >
> > Opening index @ index/
> >
> > Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> > 1 of 1: name=_0 docCount=15730
> > version=4.0.0.2
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> > Now run this little program
> >
> > public static void main(final String[] _args) throws Exception {
> > File index = new File(_args[0]); IndexWriterConfig iwcfg = new
> > IndexWriterConfig(Version.LUCENE_4_10_0,
> > new StandardAnalyzer());
> >
> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
> > Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
> > IndexWriter iw = new IndexWriter(d, iwcfg); Document doc1 = new
> > Document(); doc1.add(new StringField("type", "test", Field.Store.NO));
> > iw.addDocument(doc1); iw.close();
> > }
> >
> > and it fails with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > ) at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
> > at
> >
> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
> > 1017)
> > at
> > org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
> > at
> >
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.
> > j
> > ava:3062)
> > at
> > org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:31
> > 69
> > )
> > at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
> > at t.main(t.java:25)
> >
> > and when run CheckIndex again get
> >
> >
> > Opening index @ index/
> >
> > ERROR: could not read any segments file in directory
> > java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
> > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> > at
> >
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
> > java:176)
> > at java.nio.channels.FileChannel.open(FileChannel.java:287)
> > at java.nio.channels.FileChannel.open(FileChannel.java:334)
> > at
> >
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
> > 6)
> > at
> >
> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
> > ne40SegmentInfoReader.java:52)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
> > at
> > org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:913)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:759)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
> > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
> > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
> >
> > which is true
> >
> > total 1032
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> >
> > I don't recall the origins of this index but it may well have been
> > created in the distant past and been upgraded, explicitly or automatically,
> along the way.
> > Although evidently not for a while.
> >
> >
> > Running the same test with lucene 4.9.0 and minimal mods to the
> > program runs to successful completion. Here's the CheckIndex output:
> >
> >
> > Opening index @ index-4.9.updated/
> >
> > Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9]
> format=
> > 1 of 2: name=_0 docCount=15730
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > 2 of 2: name=_1 docCount=1
> > codec=Lucene49
> > compound=true
> > numFiles=3
> > size (MB)=0.001
> > diagnostics = {timestamp=1410281698360, os=Linux,
> > os.version=3.1.0-1.2- desktop, source=flush, lucene.version=4.9.0
> > 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
> > java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [1 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
> > test: stored fields.......OK [0 total field count; avg 0 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> >
> >
> > --
> > Ian.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Uwe Schindler <uw...@thetaphi.de>.
If you want to upgrade the index, you may try to run IndexUpgrader on Lucene 4.9, to have it up to date. But Index upgrading may fail because of the BETA-Status of the original creator.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Wednesday, September 10, 2014 1:42 PM
> To: java-user@lucene.apache.org
> Subject: RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> SegmentInfo unless codec is Lucene3x (got: Lucene40)
>
> Hi Ian,
>
> this index was created with the BETA version of Lucene 4.0:
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
>
> "4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not a
> supported version and may not open correctly. In Lucene 4.10 we changed
> version handling and parsing version numbers a bit, so this may be the cause
> for the error.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Ian Lea [mailto:ian.lea@gmail.com]
> > Sent: Wednesday, September 10, 2014 1:01 PM
> > To: java-user@lucene.apache.org
> > Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> > SegmentInfo unless codec is Lucene3x (got: Lucene40)
> >
> > Hi
> >
> >
> > On running a quick test after a handful of minor code changes to deal
> > with
> > 4.10 deprecations, a program that updates an existing index failed
> > with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > )
> >
> > and along the way did something to the index to make it unusable.
> >
> > Digging a bit deeper and working on a different old test index that
> > was lying around, and taking a backup first this time, this is reproducible.
> >
> > The working index:
> >
> > total 1036
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> > and output from 4.10 CheckIndex
> >
> > Opening index @ index/
> >
> > Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> > 1 of 1: name=_0 docCount=15730
> > version=4.0.0.2
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> > Now run this little program
> >
> > public static void main(final String[] _args) throws Exception {
> > File index = new File(_args[0]); IndexWriterConfig iwcfg = new
> > IndexWriterConfig(Version.LUCENE_4_10_0,
> > new StandardAnalyzer());
> >
> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
> > Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
> > IndexWriter iw = new IndexWriter(d, iwcfg); Document doc1 = new
> > Document(); doc1.add(new StringField("type", "test", Field.Store.NO));
> > iw.addDocument(doc1); iw.close();
> > }
> >
> > and it fails with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > ) at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
> > at
> >
> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
> > 1017)
> > at
> > org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
> > at
> >
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.
> > j
> > ava:3062)
> > at
> > org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:31
> > 69
> > )
> > at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
> > at t.main(t.java:25)
> >
> > and when run CheckIndex again get
> >
> >
> > Opening index @ index/
> >
> > ERROR: could not read any segments file in directory
> > java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
> > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> > at
> >
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
> > java:176)
> > at java.nio.channels.FileChannel.open(FileChannel.java:287)
> > at java.nio.channels.FileChannel.open(FileChannel.java:334)
> > at
> >
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
> > 6)
> > at
> >
> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
> > ne40SegmentInfoReader.java:52)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
> > at
> > org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:913)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:759)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
> > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
> > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
> >
> > which is true
> >
> > total 1032
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> >
> > I don't recall the origins of this index but it may well have been
> > created in the distant past and been upgraded, explicitly or automatically,
> along the way.
> > Although evidently not for a while.
> >
> >
> > Running the same test with lucene 4.9.0 and minimal mods to the
> > program runs to successful completion. Here's the CheckIndex output:
> >
> >
> > Opening index @ index-4.9.updated/
> >
> > Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9]
> format=
> > 1 of 2: name=_0 docCount=15730
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > 2 of 2: name=_1 docCount=1
> > codec=Lucene49
> > compound=true
> > numFiles=3
> > size (MB)=0.001
> > diagnostics = {timestamp=1410281698360, os=Linux,
> > os.version=3.1.0-1.2- desktop, source=flush, lucene.version=4.9.0
> > 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
> > java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [1 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
> > test: stored fields.......OK [0 total field count; avg 0 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> >
> >
> > --
> > Ian.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: 4.10.0: java.lang.IllegalStateException: cannot write 3x
SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Robert Muir <rc...@gmail.com>.
Ian, its a supported version. It wouldnt matter if its 4.0 alpha or
beta anyway, because we support index back compat for those.
In your case, its actually the final version. I will open an issue.
Thank you for reporting this!
On Wed, Sep 10, 2014 at 7:54 AM, Ian Lea <ia...@gmail.com> wrote:
> Yes, quite possible. I do sometimes download and test beta versions.
>
> This isn't really a problem for me - it has only happened on test
> indexes that I don't care about, but there might be live indexes out
> there that are also affected and having them made unusable would be
> undesirable, to put it mildly. A message saying "Unsupported version"
> would be much better.
>
>
> --
> Ian.
>
>
> On Wed, Sep 10, 2014 at 12:41 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Hi Ian,
>>
>> this index was created with the BETA version of Lucene 4.0:
>>
>> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
>> 1 of 1: name=_0 docCount=15730
>>
>> "4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not a supported version and may not open correctly. In Lucene 4.10 we changed version handling and parsing version numbers a bit, so this may be the cause for the error.
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: Ian Lea [mailto:ian.lea@gmail.com]
>>> Sent: Wednesday, September 10, 2014 1:01 PM
>>> To: java-user@lucene.apache.org
>>> Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo
>>> unless codec is Lucene3x (got: Lucene40)
>>>
>>> Hi
>>>
>>>
>>> On running a quick test after a handful of minor code changes to deal with
>>> 4.10 deprecations, a program that updates an existing index failed with
>>>
>>> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
>>> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
>>> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>>>
>>> and along the way did something to the index to make it unusable.
>>>
>>> Digging a bit deeper and working on a different old test index that was lying
>>> around, and taking a backup first this time, this is reproducible.
>>>
>>> The working index:
>>>
>>> total 1036
>>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>>> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
>>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>>
>>> and output from 4.10 CheckIndex
>>>
>>> Opening index @ index/
>>>
>>> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
>>> 1 of 1: name=_0 docCount=15730
>>> version=4.0.0.2
>>> codec=Lucene40
>>> compound=false
>>> numFiles=7
>>> size (MB)=0.987
>>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
>>> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
>>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>>> no deletions
>>> test: open reader.........OK
>>> test: check integrity.....OK
>>> test: check live docs.....OK
>>> test: fields..............OK [13 fields]
>>> test: field norms.........OK [0 fields]
>>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
>>> tokens]
>>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>>> fields per doc]
>>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>>
>>> No problems were detected with this index.
>>>
>>>
>>> Now run this little program
>>>
>>> public static void main(final String[] _args) throws Exception { File index =
>>> new File(_args[0]); IndexWriterConfig iwcfg = new
>>> IndexWriterConfig(Version.LUCENE_4_10_0,
>>> new StandardAnalyzer());
>>> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
>>> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
>>> IndexWriter iw = new IndexWriter(d, iwcfg); Document doc1 = new
>>> Document(); doc1.add(new StringField("type", "test", Field.Store.NO));
>>> iw.addDocument(doc1); iw.close();
>>> }
>>>
>>> and it fails with
>>>
>>> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
>>> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
>>> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>>> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
>>> at
>>> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
>>> 1017)
>>> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
>>> at
>>> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.j
>>> ava:3062)
>>> at
>>> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169
>>> )
>>> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
>>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
>>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
>>> at t.main(t.java:25)
>>>
>>> and when run CheckIndex again get
>>>
>>>
>>> Opening index @ index/
>>>
>>> ERROR: could not read any segments file in directory
>>> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
>>> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>>> at
>>> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
>>> java:176)
>>> at java.nio.channels.FileChannel.open(FileChannel.java:287)
>>> at java.nio.channels.FileChannel.open(FileChannel.java:334)
>>> at
>>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
>>> 6)
>>> at
>>> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
>>> ne40SegmentInfoReader.java:52)
>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
>>> at
>>> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
>>> at
>>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
>>> s.java:913)
>>> at
>>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
>>> s.java:759)
>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
>>> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
>>> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>>>
>>> which is true
>>>
>>> total 1032
>>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>>
>>>
>>> I don't recall the origins of this index but it may well have been created in the
>>> distant past and been upgraded, explicitly or automatically, along the way.
>>> Although evidently not for a while.
>>>
>>>
>>> Running the same test with lucene 4.9.0 and minimal mods to the program
>>> runs to successful completion. Here's the CheckIndex output:
>>>
>>>
>>> Opening index @ index-4.9.updated/
>>>
>>> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
>>> 1 of 2: name=_0 docCount=15730
>>> codec=Lucene40
>>> compound=false
>>> numFiles=7
>>> size (MB)=0.987
>>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
>>> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
>>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>>> no deletions
>>> test: open reader.........OK
>>> test: check integrity.....OK
>>> test: check live docs.....OK
>>> test: fields..............OK [13 fields]
>>> test: field norms.........OK [0 fields]
>>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
>>> tokens]
>>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>>> fields per doc]
>>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>>
>>> 2 of 2: name=_1 docCount=1
>>> codec=Lucene49
>>> compound=true
>>> numFiles=3
>>> size (MB)=0.001
>>> diagnostics = {timestamp=1410281698360, os=Linux, os.version=3.1.0-1.2-
>>> desktop, source=flush, lucene.version=4.9.0
>>> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64, java.version=1.7.0_10,
>>> java.vendor=Oracle Corporation}
>>> no deletions
>>> test: open reader.........OK
>>> test: check integrity.....OK
>>> test: check live docs.....OK
>>> test: fields..............OK [1 fields]
>>> test: field norms.........OK [0 fields]
>>> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
>>> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
>>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>>> fields per doc]
>>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>>
>>> No problems were detected with this index.
>>>
>>>
>>>
>>>
>>> --
>>> Ian.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: 4.10.0: java.lang.IllegalStateException: cannot write 3x
SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Ian Lea <ia...@gmail.com>.
Yes, quite possible. I do sometimes download and test beta versions.
This isn't really a problem for me - it has only happened on test
indexes that I don't care about, but there might be live indexes out
there that are also affected and having them made unusable would be
undesirable, to put it mildly. A message saying "Unsupported version"
would be much better.
--
Ian.
On Wed, Sep 10, 2014 at 12:41 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hi Ian,
>
> this index was created with the BETA version of Lucene 4.0:
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
>
> "4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not a supported version and may not open correctly. In Lucene 4.10 we changed version handling and parsing version numbers a bit, so this may be the cause for the error.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Ian Lea [mailto:ian.lea@gmail.com]
>> Sent: Wednesday, September 10, 2014 1:01 PM
>> To: java-user@lucene.apache.org
>> Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo
>> unless codec is Lucene3x (got: Lucene40)
>>
>> Hi
>>
>>
>> On running a quick test after a handful of minor code changes to deal with
>> 4.10 deprecations, a program that updates an existing index failed with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
>> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
>> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>>
>> and along the way did something to the index to make it unusable.
>>
>> Digging a bit deeper and working on a different old test index that was lying
>> around, and taking a backup first this time, this is reproducible.
>>
>> The working index:
>>
>> total 1036
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>> and output from 4.10 CheckIndex
>>
>> Opening index @ index/
>>
>> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
>> 1 of 1: name=_0 docCount=15730
>> version=4.0.0.2
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
>> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
>> tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>> fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>> Now run this little program
>>
>> public static void main(final String[] _args) throws Exception { File index =
>> new File(_args[0]); IndexWriterConfig iwcfg = new
>> IndexWriterConfig(Version.LUCENE_4_10_0,
>> new StandardAnalyzer());
>> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
>> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
>> IndexWriter iw = new IndexWriter(d, iwcfg); Document doc1 = new
>> Document(); doc1.add(new StringField("type", "test", Field.Store.NO));
>> iw.addDocument(doc1); iw.close();
>> }
>>
>> and it fails with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
>> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
>> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
>> at
>> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
>> 1017)
>> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
>> at
>> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.j
>> ava:3062)
>> at
>> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169
>> )
>> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
>> at t.main(t.java:25)
>>
>> and when run CheckIndex again get
>>
>>
>> Opening index @ index/
>>
>> ERROR: could not read any segments file in directory
>> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
>> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>> at
>> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
>> java:176)
>> at java.nio.channels.FileChannel.open(FileChannel.java:287)
>> at java.nio.channels.FileChannel.open(FileChannel.java:334)
>> at
>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
>> 6)
>> at
>> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
>> ne40SegmentInfoReader.java:52)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
>> at
>> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
>> at
>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
>> s.java:913)
>> at
>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
>> s.java:759)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
>> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
>> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>>
>> which is true
>>
>> total 1032
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>>
>> I don't recall the origins of this index but it may well have been created in the
>> distant past and been upgraded, explicitly or automatically, along the way.
>> Although evidently not for a while.
>>
>>
>> Running the same test with lucene 4.9.0 and minimal mods to the program
>> runs to successful completion. Here's the CheckIndex output:
>>
>>
>> Opening index @ index-4.9.updated/
>>
>> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
>> 1 of 2: name=_0 docCount=15730
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
>> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
>> tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>> fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> 2 of 2: name=_1 docCount=1
>> codec=Lucene49
>> compound=true
>> numFiles=3
>> size (MB)=0.001
>> diagnostics = {timestamp=1410281698360, os=Linux, os.version=3.1.0-1.2-
>> desktop, source=flush, lucene.version=4.9.0
>> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64, java.version=1.7.0_10,
>> java.vendor=Oracle Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [1 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
>> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
>> fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
>> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>>
>>
>> --
>> Ian.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Ian,
this index was created with the BETA version of Lucene 4.0:
Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
1 of 1: name=_0 docCount=15730
"4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not a supported version and may not open correctly. In Lucene 4.10 we changed version handling and parsing version numbers a bit, so this may be the cause for the error.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Ian Lea [mailto:ian.lea@gmail.com]
> Sent: Wednesday, September 10, 2014 1:01 PM
> To: java-user@lucene.apache.org
> Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x SegmentInfo
> unless codec is Lucene3x (got: Lucene40)
>
> Hi
>
>
> On running a quick test after a handful of minor code changes to deal with
> 4.10 deprecations, a program that updates an existing index failed with
>
> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>
> and along the way did something to the index to make it unusable.
>
> Digging a bit deeper and working on a different old test index that was lying
> around, and taking a backup first this time, this is reproducible.
>
> The working index:
>
> total 1036
> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>
> and output from 4.10 CheckIndex
>
> Opening index @ index/
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
> version=4.0.0.2
> codec=Lucene40
> compound=false
> numFiles=7
> size (MB)=0.987
> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
> java.version=1.7.0_10, java.vendor=Oracle Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [13 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
> tokens]
> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
> fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> No problems were detected with this index.
>
>
> Now run this little program
>
> public static void main(final String[] _args) throws Exception { File index =
> new File(_args[0]); IndexWriterConfig iwcfg = new
> IndexWriterConfig(Version.LUCENE_4_10_0,
> new StandardAnalyzer());
> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
> IndexWriter iw = new IndexWriter(d, iwcfg); Document doc1 = new
> Document(); doc1.add(new StringField("type", "test", Field.Store.NO));
> iw.addDocument(doc1); iw.close();
> }
>
> and it fails with
>
> Exception in thread "main" java.lang.IllegalStateException: cannot write 3x
> SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
> at
> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
> 1017)
> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
> at
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.j
> ava:3062)
> at
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169
> )
> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
> at t.main(t.java:25)
>
> and when run CheckIndex again get
>
>
> Opening index @ index/
>
> ERROR: could not read any segments file in directory
> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
> java:176)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:334)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
> 6)
> at
> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
> ne40SegmentInfoReader.java:52)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
> at
> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> s.java:913)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> s.java:759)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>
> which is true
>
> total 1032
> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>
>
> I don't recall the origins of this index but it may well have been created in the
> distant past and been upgraded, explicitly or automatically, along the way.
> Although evidently not for a while.
>
>
> Running the same test with lucene 4.9.0 and minimal mods to the program
> runs to successful completion. Here's the CheckIndex output:
>
>
> Opening index @ index-4.9.updated/
>
> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
> 1 of 2: name=_0 docCount=15730
> codec=Lucene40
> compound=false
> numFiles=7
> size (MB)=0.987
> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, source=flush,
> lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12, os.arch=amd64,
> java.version=1.7.0_10, java.vendor=Oracle Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [13 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs pairs; 139382
> tokens]
> test: stored fields.......OK [15730 total field count; avg 1 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
> fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> 2 of 2: name=_1 docCount=1
> codec=Lucene49
> compound=true
> numFiles=3
> size (MB)=0.001
> diagnostics = {timestamp=1410281698360, os=Linux, os.version=3.1.0-1.2-
> desktop, source=flush, lucene.version=4.9.0
> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64, java.version=1.7.0_10,
> java.vendor=Oracle Corporation}
> no deletions
> test: open reader.........OK
> test: check integrity.....OK
> test: check live docs.....OK
> test: fields..............OK [1 fields]
> test: field norms.........OK [0 fields]
> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
> test: term vectors........OK [0 total vector count; avg 0 term/freq vector
> fields per doc]
> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0
> SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>
> No problems were detected with this index.
>
>
>
>
> --
> Ian.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org