You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Sven Teichmann <s....@s4ip.de> on 2015/06/16 10:33:35 UTC
IndexFormatTooOldException while upgrading Lucene 4.10 index to 5.2
Today I tried to upgrade a Lucene 4.10 index to 5.2 using the IndexUpgrader.
This throws the following exception:
Exception in thread "Lucene Merge Thread #0"
org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.IndexFormatTooOldException: Format version is
not supported (resource
BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_201506/vol2/_0_Direct_0.dvdm"))):
2 (needs to be between 3 and 3). This version of Lucene only supports
indexes created with release 4.0 and later.
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:673)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:641)
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format
version is not supported (resource
BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_201506/vol2/_0_Direct_0.dvdm"))):
2 (needs to be between 3 and 3). This version of Lucene only supports
indexes created with release 4.0 and later.
at
org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:214)
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:199)
at org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:256)
at
org.apache.lucene.codecs.memory.DirectDocValuesProducer.<init>(DirectDocValuesProducer.java:126)
at
org.apache.lucene.codecs.memory.DirectDocValuesFormat.fieldsProducer(DirectDocValuesFormat.java:76)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:268)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:358)
at
org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
at
org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:67)
at
org.apache.lucene.index.SegmentReader.initDocValuesProducer(SegmentReader.java:141)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:81)
at
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:132)
at
org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:604)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3893)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3543)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:581)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:619)
We use DirectDocValuesFormat for two fields and I think that this is the
problem.
Any ideas how to upgrade the index to 5.2 anyway?
Thanks for your help!
Sven
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: IndexFormatTooOldException while upgrading Lucene 4.10 index to 5.2
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
Sorry, I just noticed that you are using DirectDocValues format, which was experimental. You cannot directly upgrade this to 5.0. I would suggest to convert your index to use the default codec in 4.10 (e.g., by using addIndexes to a new index directory, or by forceMerge with an IndexWriter that has default codec).
After that you can read the 4.10 index with 5.0 using lucene-backward-codecs.jar in classpath.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Tuesday, June 16, 2015 11:35 AM
> To: java-user@lucene.apache.org
> Subject: RE: IndexFormatTooOldException while upgrading Lucene 4.10 index
> to 5.2
>
> Hi,
>
> you need to add the JAR file lucene-backward-codec.jar to the classpath (or
> add it via Maven). It contains the codecs to read pre-5.0 versions.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Sven Teichmann [mailto:s.teichmann@s4ip.de]
> > Sent: Tuesday, June 16, 2015 10:34 AM
> > To: java-user@lucene.apache.org
> > Subject: IndexFormatTooOldException while upgrading Lucene 4.10 index
> > to
> > 5.2
> >
> > Today I tried to upgrade a Lucene 4.10 index to 5.2 using the
> IndexUpgrader.
> > This throws the following exception:
> >
> > Exception in thread "Lucene Merge Thread #0"
> > org.apache.lucene.index.MergePolicy$MergeException:
> > org.apache.lucene.index.IndexFormatTooOldException: Format version is
> > not supported (resource
> >
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_
> > 201506/vol2/_0_Direct_0.dvdm"))):
> > 2 (needs to be between 3 and 3). This version of Lucene only supports
> > indexes created with release 4.0 and later.
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeExceptio
> > n(ConcurrentMergeScheduler.java:673)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Co
> > ncurrentMergeScheduler.java:641)
> > Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format
> > version is not supported (resource
> >
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_
> > 201506/vol2/_0_Direct_0.dvdm"))):
> > 2 (needs to be between 3 and 3). This version of Lucene only supports
> > indexes created with release 4.0 and later.
> > at
> >
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:2
> > 14)
> > at
> > org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:199)
> > at
> >
> org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:256)
> > at
> >
> org.apache.lucene.codecs.memory.DirectDocValuesProducer.<init>(DirectD
> > ocValuesProducer.java:126)
> > at
> >
> org.apache.lucene.codecs.memory.DirectDocValuesFormat.fieldsProducer(D
> > irectDocValuesFormat.java:76)
> > at
> >
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.
> > <init>(PerFieldDocValuesFormat.java:268)
> > at
> > org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProduc
> > er
> > (PerFieldDocValuesFormat.java:358)
> > at
> >
> org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(Segm
> > entDocValues.java:51)
> > at
> >
> org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(Segme
> > ntDocValues.java:67)
> > at
> >
> org.apache.lucene.index.SegmentReader.initDocValuesProducer(SegmentR
> > eader.java:141)
> > at
> > org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:81)
> > at
> >
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdat
> > es.java:132)
> > at
> >
> org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(Readers
> > AndUpdates.java:604)
> > at
> >
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3893)
> > at
> > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3543)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(Concurrent
> > MergeScheduler.java:581)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Co
> > ncurrentMergeScheduler.java:619)
> >
> >
> > We use DirectDocValuesFormat for two fields and I think that this is
> > the problem.
> >
> > Any ideas how to upgrade the index to 5.2 anyway?
> >
> > Thanks for your help!
> >
> > Sven
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: IndexFormatTooOldException while upgrading Lucene 4.10 index to 5.2
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
you need to add the JAR file lucene-backward-codec.jar to the classpath (or add it via Maven). It contains the codecs to read pre-5.0 versions.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Sven Teichmann [mailto:s.teichmann@s4ip.de]
> Sent: Tuesday, June 16, 2015 10:34 AM
> To: java-user@lucene.apache.org
> Subject: IndexFormatTooOldException while upgrading Lucene 4.10 index to
> 5.2
>
> Today I tried to upgrade a Lucene 4.10 index to 5.2 using the IndexUpgrader.
> This throws the following exception:
>
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.IndexFormatTooOldException: Format version is
> not supported (resource
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_
> 201506/vol2/_0_Direct_0.dvdm"))):
> 2 (needs to be between 3 and 3). This version of Lucene only supports
> indexes created with release 4.0 and later.
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeExceptio
> n(ConcurrentMergeScheduler.java:673)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Co
> ncurrentMergeScheduler.java:641)
> Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format
> version is not supported (resource
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_
> 201506/vol2/_0_Direct_0.dvdm"))):
> 2 (needs to be between 3 and 3). This version of Lucene only supports
> indexes created with release 4.0 and later.
> at
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:2
> 14)
> at
> org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:199)
> at
> org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:256)
> at
> org.apache.lucene.codecs.memory.DirectDocValuesProducer.<init>(DirectD
> ocValuesProducer.java:126)
> at
> org.apache.lucene.codecs.memory.DirectDocValuesFormat.fieldsProducer(D
> irectDocValuesFormat.java:76)
> at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.
> <init>(PerFieldDocValuesFormat.java:268)
> at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer
> (PerFieldDocValuesFormat.java:358)
> at
> org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(Segm
> entDocValues.java:51)
> at
> org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(Segme
> ntDocValues.java:67)
> at
> org.apache.lucene.index.SegmentReader.initDocValuesProducer(SegmentR
> eader.java:141)
> at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:81)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdat
> es.java:132)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(Readers
> AndUpdates.java:604)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3893)
> at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3543)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(Concurrent
> MergeScheduler.java:581)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Co
> ncurrentMergeScheduler.java:619)
>
>
> We use DirectDocValuesFormat for two fields and I think that this is the
> problem.
>
> Any ideas how to upgrade the index to 5.2 anyway?
>
> Thanks for your help!
>
> Sven
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexFormatTooOldException while upgrading Lucene 4.10 index to 5.2
Posted by Michael McCandless <lu...@mikemccandless.com>.
Indeed DirectDocValuesFormat had a back-compat break between 4.x and
5.x ... this is expected/allowed (it is an experimental format).
You should be able to do something like this to more forwards:
* In 4.x, open a new IndexWriter, using the default Codec (which is
back-compat)
* Use addIndexes to add in your indices that use DirectDocValuesFormat
* Upgrade to 5.x
* Open a new IndexWriter, with your Codec that uses DirectDocValuesFormat
* Use addIndexes to add the index
Mike McCandless
http://blog.mikemccandless.com
On Tue, Jun 16, 2015 at 4:33 AM, Sven Teichmann <s....@s4ip.de> wrote:
> Today I tried to upgrade a Lucene 4.10 index to 5.2 using the IndexUpgrader.
> This throws the following exception:
>
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.IndexFormatTooOldException: Format version is
> not supported (resource
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_201506/vol2/_0_Direct_0.dvdm"))):
> 2 (needs to be between 3 and 3). This version of Lucene only supports
> indexes created with release 4.0 and later.
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:673)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:641)
> Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format
> version is not supported (resource
> BufferedChecksumIndexInput(MMapIndexInput(path="/data/title_lu5_en_201506/vol2/_0_Direct_0.dvdm"))):
> 2 (needs to be between 3 and 3). This version of Lucene only supports
> indexes created with release 4.0 and later.
> at
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:214)
> at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:199)
> at org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:256)
> at
> org.apache.lucene.codecs.memory.DirectDocValuesProducer.<init>(DirectDocValuesProducer.java:126)
> at
> org.apache.lucene.codecs.memory.DirectDocValuesFormat.fieldsProducer(DirectDocValuesFormat.java:76)
> at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:268)
> at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:358)
> at
> org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
> at
> org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:67)
> at
> org.apache.lucene.index.SegmentReader.initDocValuesProducer(SegmentReader.java:141)
> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:81)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:132)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:604)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3893)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3543)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:581)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:619)
>
>
> We use DirectDocValuesFormat for two fields and I think that this is the
> problem.
>
> Any ideas how to upgrade the index to 5.2 anyway?
>
> Thanks for your help!
>
> Sven
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org