Merge problem with lucene 3 and 4 indices

[Harry Hight <ne...@gmail.com>]

We have a process that builds small indices, and merges them into the
master, and are in the process of going from solr 3.5 to solr 4.3. So,
during this process we are going to have to merge indices built with solr 3
with ones built with solr 4.

I'm running into a problem with an index built from that process. It was
merged from a set of solr 3 indices by solr 4 code, but it wrote a solr 3
segment.

Searching on the index works fine, however, this code:
Directory[] indexes = new Directory[1];
indexes[0]  = new NIOFSDirectory(new File(dir));
writer.addIndexes(indexes);

fails in addIndexes() with
Exception in thread "main" java.io.FileNotFoundException: _6.tis
        at
org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:266)
        at
org.apache.lucene.index.SegmentInfoPerCommit.sizeInBytes(SegmentInfoPerCommit.java:88)
        at
org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2319)

If I rename the files from _0 to _6, it fails with
Exception in thread "main" java.io.FileNotFoundException: <path omitted>/_
0.si (No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
        at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:410)
        at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDirectory.java:123)
        at
org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:80)
        at
org.apache.lucene.codecs.lucene3x.Lucene3xSegmentInfoReader.read(Lucene3xSegmentInfoReader.java:103)
        at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:301)
        at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:347)
        at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
        at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:630)
        at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:343)

If I copy the _0 files to _6, the merge works fine, but I don't understand
why its trying to find the _6 segment in the first place.

Hex dump of the segments_1 file:
0000000: 3fd7 6c17 0873 6567 6d65 6e74 7300 0000  ?.l..segments...
0000010: 0000 0000 0000 0000 0300 0000 0100 0000  ................
0000020: 0102 5f30 084c 7563 656e 6533 78ff ffff  .._0.Lucene3x...
0000030: ffff ffff ff00 0000 0000 0000 0000 0000  ................
0000040: 00f6 90cf 88                             .....

Index directory:
drwxrwxr-x 2 hhight general 4.0K Sep 18 17:19 ./
drwxrwxr-x 3 hhight general 4.0K Sep 18 13:46 ../
-rw-rw-r-- 1 hhight general  34M Sep 18 13:25 _0.fdt
-rw-rw-r-- 1 hhight general  26K Sep 18 13:25 _0.fdx
-rw-rw-r-- 1 hhight general 2.7K Sep 18 13:25 _0.fnm
-rw-rw-r-- 1 hhight general 3.9M Sep 18 13:25 _0.frq
-rw-rw-r-- 1 hhight general 195K Sep 18 13:25 _0.nrm
-rw-rw-r-- 1 hhight general 8.5M Sep 18 13:25 _0.prx
-rw-rw-r-- 1 hhight general  343 Sep 18 13:25 _0.si
-rw-rw-r-- 1 hhight general 118K Sep 18 13:25 _0.tii
-rw-rw-r-- 1 hhight general 8.5M Sep 18 13:25 _0.tis
-rw-rw-r-- 1 hhight general   29 Sep 18 13:25 _0_upgraded.si
-rw-rw-r-- 1 hhight general   69 Sep 18 13:25 segments_1
-rw-rw-r-- 1 hhight general   20 Sep 18 13:25 segments.gen


Any suggestions on the cause of this?