You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Uwe Schindler <uw...@thetaphi.de> on 2009/11/17 23:25:03 UTC
Lucene Java 3.0.0 RC1 now available for testing
Hello Lucene users,
On behalf of the Lucene dev community (a growing community far larger than
just the committers) I would like to announce the first release candidate
for Lucene Java 3.0.
Please download and check it out - take it for a spin and kick the tires. If
all goes well, we hope to release the final version of Lucene 3.0 in a
little over a week.
The new version is mostly a cleanup release without any new features. All
deprecations targeted to be removed in version 3.0 were removed. If you are
upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
warnings in your code base to be able to recompile against this version.
This is the first Lucene release with Java 5 as a minimum requirement. The
API was cleaned up to make use of Java 5's generics, varargs, enums, and
autoboxing. New users of Lucene are advised to use this version for new
developments, because it has a clean, type safe new API. Upgrading users can
now remove unnecessary casts and add generics to their code, too. If you
have not upgraded your installation to Java 5, please read the file
JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
3.0, it will also happen with any previous release when you upgrade your
Java environment).
Lucene 3.0 has some changes regarding compressed fields: 2.9 already
deprecated compressed fields; support for them was removed now. Lucene 3.0
is still able to read indexes with compressed fields, but as soon as merges
occur or the index is optimized, all compressed fields are decompressed and
converted to Field.Store.YES. Because of this, indexes with compressed
fields can suddenly get larger.
While we generally try and maintain full backwards compatibility between
major versions, Lucene 3.0 has some minor breaks, mostly related to
deprecation removal, pointed out in the 'Changes in backwards compatibility
policy' section of CHANGES.txt. Notable are:
- IndexReader.open(Directory) now opens in read-only mode per default (this
method was deprecated because of that in 2.9). The same occurs to
IndexSearcher.
- Already started in 2.9, core TokenStreams are now made final to enforce
the decorator pattern.
- If you interrupt an IndexWriter merge thread, IndexWriter now throws an
unchecked ThreadInterruptedException that extends RuntimeException and
clears the interrupt status.
Also, remember that this is a release candidate, and not the final Lucene
3.0 release.
You can find the full list of changes here:
HTML version:
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
hanges.html
Text version:
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
hanges.txt
Changes have also occurred in Lucene's contrib area:
HTML version:
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
ontrib-Changes.html
Text version:
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
ontrib-Changes.txt
Download release candidate 1 here:
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
Be sure to report back with any issues you find! Look especially for faults
in generification of public APIs (like missing wildcards,...).
Thanks,
Uwe Schindler
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
RE: Lucene Java 3.0.0 RC1 now available for testing
Posted by Uwe Schindler <uw...@thetaphi.de>.
There are already some proposals of reforming the whole Document/Field API
because it does not match a Full Text Search engine using an Inverted Index.
Stored fields and indexed fields should not be mixed together. The problem
then disappears, because you are forced to split between indexing and
storing so you have to take care in separate about storing and indexing.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Glen Newton [mailto:glen.newton@gmail.com]
> Sent: Wednesday, November 18, 2009 3:24 PM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>
> Yes, I would agree with you on the surprise aspect. :-)
>
> But you suggest hiding complexity, and being in control and having
> transparency are mutually exclusive, which isn't necesarily the case.
>
> I think I can live with the decisions made. :-)
> If I can think of a viable and complete alternative, I'll run it by
> the community.
>
> thanks,
> Glen
>
> 2009/11/18 Otis Gospodnetic <ot...@yahoo.com>:
> > Well, I think some people will be for hiding complexity, while others
> will be for being in control and having transparency. Think how surprised
> one would be to find 1 extra field in his index, say when looking at their
> index with Luke. :)
> > Otis
> > --
> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> >
> >
> >
> > ----- Original Message ----
> >> From: Glen Newton <gl...@gmail.com>
> >> To: java-user@lucene.apache.org
> >> Sent: Tue, November 17, 2009 10:53:01 PM
> >> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
> >>
> >> I understand the reasons, but - if I may ask so late in the game - was
> >> this the best way to do this?
> >>
> >> From a user (developer) perspective, this is an implementation issue.
> >> Couldn't this have been done behind the scenes, so that when I asked
> >> for Field.Index.ANALYZED && Field.Store.COMPRESS, instead of what
> >> previously happened (and was variously problematic), two fields were
> >> transparently created, one being binary compressed stored and the
> >> other being indexed only? The Field API could hide all of this
> >> complexity, using one underlying Field when I use Field.getString()
> >> (compressed stored one), using the other when I use Field.setBoost()
> >> (the indexed one) and both when I call Field.setValue(). This might
> >> have less impact on developers and be less disruptive on API changes.
> >> Oh, some naming convention could handle the underlying Fields.
> >>
> >> A little complicated I agree.
> >>
> >> Again, apologies to those who worked hard on these changes: my fault
> >> for not noticing this sooner (I hadn't started moving my code to 2.9
> >> from 2.4 so I hadn't read the deprecation signs).
> >>
> >> thanks,
> >>
> >> Glen
> >>
> >>
> >>
> >> 2009/11/17 Mark Miller :
> >> > Here is some of the history:
> >> >
> >> > https://issues.apache.org/jira/browse/LUCENE-652
> >> > https://issues.apache.org/jira/browse/LUCENE-1960
> >> >
> >> > Glen Newton wrote:
> >> >> Could someone send me where the rationale for the removal of
> >> >> COMPRESSED fields is? I've looked at
> >> >>
> >> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
> >> >> but it is a little light on the 'why' of this change.
> >> >>
> >> >> My fault - of course - for not paying attention.
> >> >>
> >> >> thanks,
> >> >> Glen
> >> >>
> >> >> 2009/11/17 Uwe Schindler :
> >> >>
> >> >>> Hello Lucene users,
> >> >>>
> >> >>>
> >> >>>
> >> >>> On behalf of the Lucene dev community (a growing community far
> larger than
> >> >>> just the committers) I would like to announce the first release
> candidate
> >> >>> for Lucene Java 3.0.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Please download and check it out - take it for a spin and kick the
> tires. If
> >> >>> all goes well, we hope to release the final version of Lucene 3.0
> in a
> >> >>> little over a week.
> >> >>>
> >> >>>
> >> >>>
> >> >>> The new version is mostly a cleanup release without any new
> features. All
> >> >>> deprecations targeted to be removed in version 3.0 were removed. If
> you are
> >> >>> upgrading from version 2.9.1 of Lucene, you have to fix all
> deprecation
> >> >>> warnings in your code base to be able to recompile against this
> version.
> >> >>>
> >> >>>
> >> >>>
> >> >>> This is the first Lucene release with Java 5 as a minimum
> requirement. The
> >> >>> API was cleaned up to make use of Java 5's generics, varargs,
> enums, and
> >> >>> autoboxing. New users of Lucene are advised to use this version for
> new
> >> >>> developments, because it has a clean, type safe new API. Upgrading
> users can
> >> >>> now remove unnecessary casts and add generics to their code, too.
> If you
> >> >>> have not upgraded your installation to Java 5, please read the file
> >> >>> JRE_VERSION_MIGRATION.txt (please note that this is not related to
> Lucene
> >> >>> 3.0, it will also happen with any previous release when you upgrade
> your
> >> >>> Java environment).
> >> >>>
> >> >>>
> >> >>>
> >> >>> Lucene 3.0 has some changes regarding compressed fields: 2.9
> already
> >> >>> deprecated compressed fields; support for them was removed now.
> Lucene 3.0
> >> >>> is still able to read indexes with compressed fields, but as soon
> as merges
> >> >>> occur or the index is optimized, all compressed fields are
> decompressed and
> >> >>> converted to Field.Store.YES. Because of this, indexes with
> compressed
> >> >>> fields can suddenly get larger.
> >> >>>
> >> >>>
> >> >>>
> >> >>> While we generally try and maintain full backwards compatibility
> between
> >> >>> major versions, Lucene 3.0 has some minor breaks, mostly related to
> >> >>> deprecation removal, pointed out in the 'Changes in backwards
> compatibility
> >> >>> policy' section of CHANGES.txt. Notable are:
> >> >>>
> >> >>>
> >> >>>
> >> >>> - IndexReader.open(Directory) now opens in read-only mode per
> default (this
> >> >>> method was deprecated because of that in 2.9). The same occurs to
> >> >>> IndexSearcher.
> >> >>>
> >> >>> - Already started in 2.9, core TokenStreams are now made final to
> enforce
> >> >>> the decorator pattern.
> >> >>>
> >> >>> - If you interrupt an IndexWriter merge thread, IndexWriter now
> throws an
> >> >>> unchecked ThreadInterruptedException that extends RuntimeException
> and
> >> >>> clears the interrupt status.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Also, remember that this is a release candidate, and not the final
> Lucene
> >> >>> 3.0 release.
> >> >>>
> >> >>>
> >> >>>
> >> >>> You can find the full list of changes here:
> >> >>>
> >> >>>
> >> >>>
> >> >>> HTML version:
> >> >>>
> >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> >> >>> hanges.html
> >> >>>
> >> >>>
> >> >>>
> >> >>> Text version:
> >> >>>
> >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> >> >>> hanges.txt
> >> >>>
> >> >>>
> >> >>>
> >> >>> Changes have also occurred in Lucene's contrib area:
> >> >>>
> >> >>>
> >> >>>
> >> >>> HTML version:
> >> >>>
> >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> >> >>> ontrib-Changes.html
> >> >>>
> >> >>>
> >> >>>
> >> >>> Text version:
> >> >>>
> >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> >> >>> ontrib-Changes.txt
> >> >>>
> >> >>>
> >> >>>
> >> >>> Download release candidate 1 here:
> >> >>>
> >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
> >> >>>
> >> >>>
> >> >>>
> >> >>> Be sure to report back with any issues you find! Look especially
> for faults
> >> >>> in generification of public APIs (like missing wildcards,...).
> >> >>>
> >> >>>
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>> Uwe Schindler
> >> >>>
> >> >>>
> >> >>>
> >> >>> -----
> >> >>>
> >> >>> Uwe Schindler
> >> >>>
> >> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> >>>
> >> >>> http://www.thetaphi.de
> >> >>>
> >> >>> eMail: uwe@thetaphi.de
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > - Mark
> >> >
> >> > http://www.lucidimagination.com
> >> >
> >> >
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> -
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
>
> --
>
> -
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Glen Newton <gl...@gmail.com>.
Yes, I would agree with you on the surprise aspect. :-)
But you suggest hiding complexity, and being in control and having
transparency are mutually exclusive, which isn't necesarily the case.
I think I can live with the decisions made. :-)
If I can think of a viable and complete alternative, I'll run it by
the community.
thanks,
Glen
2009/11/18 Otis Gospodnetic <ot...@yahoo.com>:
> Well, I think some people will be for hiding complexity, while others will be for being in control and having transparency. Think how surprised one would be to find 1 extra field in his index, say when looking at their index with Luke. :)
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
>> From: Glen Newton <gl...@gmail.com>
>> To: java-user@lucene.apache.org
>> Sent: Tue, November 17, 2009 10:53:01 PM
>> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>>
>> I understand the reasons, but - if I may ask so late in the game - was
>> this the best way to do this?
>>
>> From a user (developer) perspective, this is an implementation issue.
>> Couldn't this have been done behind the scenes, so that when I asked
>> for Field.Index.ANALYZED && Field.Store.COMPRESS, instead of what
>> previously happened (and was variously problematic), two fields were
>> transparently created, one being binary compressed stored and the
>> other being indexed only? The Field API could hide all of this
>> complexity, using one underlying Field when I use Field.getString()
>> (compressed stored one), using the other when I use Field.setBoost()
>> (the indexed one) and both when I call Field.setValue(). This might
>> have less impact on developers and be less disruptive on API changes.
>> Oh, some naming convention could handle the underlying Fields.
>>
>> A little complicated I agree.
>>
>> Again, apologies to those who worked hard on these changes: my fault
>> for not noticing this sooner (I hadn't started moving my code to 2.9
>> from 2.4 so I hadn't read the deprecation signs).
>>
>> thanks,
>>
>> Glen
>>
>>
>>
>> 2009/11/17 Mark Miller :
>> > Here is some of the history:
>> >
>> > https://issues.apache.org/jira/browse/LUCENE-652
>> > https://issues.apache.org/jira/browse/LUCENE-1960
>> >
>> > Glen Newton wrote:
>> >> Could someone send me where the rationale for the removal of
>> >> COMPRESSED fields is? I've looked at
>> >>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
>> >> but it is a little light on the 'why' of this change.
>> >>
>> >> My fault - of course - for not paying attention.
>> >>
>> >> thanks,
>> >> Glen
>> >>
>> >> 2009/11/17 Uwe Schindler :
>> >>
>> >>> Hello Lucene users,
>> >>>
>> >>>
>> >>>
>> >>> On behalf of the Lucene dev community (a growing community far larger than
>> >>> just the committers) I would like to announce the first release candidate
>> >>> for Lucene Java 3.0.
>> >>>
>> >>>
>> >>>
>> >>> Please download and check it out - take it for a spin and kick the tires. If
>> >>> all goes well, we hope to release the final version of Lucene 3.0 in a
>> >>> little over a week.
>> >>>
>> >>>
>> >>>
>> >>> The new version is mostly a cleanup release without any new features. All
>> >>> deprecations targeted to be removed in version 3.0 were removed. If you are
>> >>> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
>> >>> warnings in your code base to be able to recompile against this version.
>> >>>
>> >>>
>> >>>
>> >>> This is the first Lucene release with Java 5 as a minimum requirement. The
>> >>> API was cleaned up to make use of Java 5's generics, varargs, enums, and
>> >>> autoboxing. New users of Lucene are advised to use this version for new
>> >>> developments, because it has a clean, type safe new API. Upgrading users can
>> >>> now remove unnecessary casts and add generics to their code, too. If you
>> >>> have not upgraded your installation to Java 5, please read the file
>> >>> JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
>> >>> 3.0, it will also happen with any previous release when you upgrade your
>> >>> Java environment).
>> >>>
>> >>>
>> >>>
>> >>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
>> >>> deprecated compressed fields; support for them was removed now. Lucene 3.0
>> >>> is still able to read indexes with compressed fields, but as soon as merges
>> >>> occur or the index is optimized, all compressed fields are decompressed and
>> >>> converted to Field.Store.YES. Because of this, indexes with compressed
>> >>> fields can suddenly get larger.
>> >>>
>> >>>
>> >>>
>> >>> While we generally try and maintain full backwards compatibility between
>> >>> major versions, Lucene 3.0 has some minor breaks, mostly related to
>> >>> deprecation removal, pointed out in the 'Changes in backwards compatibility
>> >>> policy' section of CHANGES.txt. Notable are:
>> >>>
>> >>>
>> >>>
>> >>> - IndexReader.open(Directory) now opens in read-only mode per default (this
>> >>> method was deprecated because of that in 2.9). The same occurs to
>> >>> IndexSearcher.
>> >>>
>> >>> - Already started in 2.9, core TokenStreams are now made final to enforce
>> >>> the decorator pattern.
>> >>>
>> >>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws an
>> >>> unchecked ThreadInterruptedException that extends RuntimeException and
>> >>> clears the interrupt status.
>> >>>
>> >>>
>> >>>
>> >>> Also, remember that this is a release candidate, and not the final Lucene
>> >>> 3.0 release.
>> >>>
>> >>>
>> >>>
>> >>> You can find the full list of changes here:
>> >>>
>> >>>
>> >>>
>> >>> HTML version:
>> >>>
>> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> >>> hanges.html
>> >>>
>> >>>
>> >>>
>> >>> Text version:
>> >>>
>> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> >>> hanges.txt
>> >>>
>> >>>
>> >>>
>> >>> Changes have also occurred in Lucene's contrib area:
>> >>>
>> >>>
>> >>>
>> >>> HTML version:
>> >>>
>> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> >>> ontrib-Changes.html
>> >>>
>> >>>
>> >>>
>> >>> Text version:
>> >>>
>> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> >>> ontrib-Changes.txt
>> >>>
>> >>>
>> >>>
>> >>> Download release candidate 1 here:
>> >>>
>> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
>> >>>
>> >>>
>> >>>
>> >>> Be sure to report back with any issues you find! Look especially for faults
>> >>> in generification of public APIs (like missing wildcards,...).
>> >>>
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Uwe Schindler
>> >>>
>> >>>
>> >>>
>> >>> -----
>> >>>
>> >>> Uwe Schindler
>> >>>
>> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> >>>
>> >>> http://www.thetaphi.de
>> >>>
>> >>> eMail: uwe@thetaphi.de
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > - Mark
>> >
>> > http://www.lucidimagination.com
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>>
>>
>> --
>>
>> -
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
-
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Well, I think some people will be for hiding complexity, while others will be for being in control and having transparency. Think how surprised one would be to find 1 extra field in his index, say when looking at their index with Luke. :)
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: Glen Newton <gl...@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Tue, November 17, 2009 10:53:01 PM
> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>
> I understand the reasons, but - if I may ask so late in the game - was
> this the best way to do this?
>
> From a user (developer) perspective, this is an implementation issue.
> Couldn't this have been done behind the scenes, so that when I asked
> for Field.Index.ANALYZED && Field.Store.COMPRESS, instead of what
> previously happened (and was variously problematic), two fields were
> transparently created, one being binary compressed stored and the
> other being indexed only? The Field API could hide all of this
> complexity, using one underlying Field when I use Field.getString()
> (compressed stored one), using the other when I use Field.setBoost()
> (the indexed one) and both when I call Field.setValue(). This might
> have less impact on developers and be less disruptive on API changes.
> Oh, some naming convention could handle the underlying Fields.
>
> A little complicated I agree.
>
> Again, apologies to those who worked hard on these changes: my fault
> for not noticing this sooner (I hadn't started moving my code to 2.9
> from 2.4 so I hadn't read the deprecation signs).
>
> thanks,
>
> Glen
>
>
>
> 2009/11/17 Mark Miller :
> > Here is some of the history:
> >
> > https://issues.apache.org/jira/browse/LUCENE-652
> > https://issues.apache.org/jira/browse/LUCENE-1960
> >
> > Glen Newton wrote:
> >> Could someone send me where the rationale for the removal of
> >> COMPRESSED fields is? I've looked at
> >>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
> >> but it is a little light on the 'why' of this change.
> >>
> >> My fault - of course - for not paying attention.
> >>
> >> thanks,
> >> Glen
> >>
> >> 2009/11/17 Uwe Schindler :
> >>
> >>> Hello Lucene users,
> >>>
> >>>
> >>>
> >>> On behalf of the Lucene dev community (a growing community far larger than
> >>> just the committers) I would like to announce the first release candidate
> >>> for Lucene Java 3.0.
> >>>
> >>>
> >>>
> >>> Please download and check it out - take it for a spin and kick the tires. If
> >>> all goes well, we hope to release the final version of Lucene 3.0 in a
> >>> little over a week.
> >>>
> >>>
> >>>
> >>> The new version is mostly a cleanup release without any new features. All
> >>> deprecations targeted to be removed in version 3.0 were removed. If you are
> >>> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
> >>> warnings in your code base to be able to recompile against this version.
> >>>
> >>>
> >>>
> >>> This is the first Lucene release with Java 5 as a minimum requirement. The
> >>> API was cleaned up to make use of Java 5's generics, varargs, enums, and
> >>> autoboxing. New users of Lucene are advised to use this version for new
> >>> developments, because it has a clean, type safe new API. Upgrading users can
> >>> now remove unnecessary casts and add generics to their code, too. If you
> >>> have not upgraded your installation to Java 5, please read the file
> >>> JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
> >>> 3.0, it will also happen with any previous release when you upgrade your
> >>> Java environment).
> >>>
> >>>
> >>>
> >>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
> >>> deprecated compressed fields; support for them was removed now. Lucene 3.0
> >>> is still able to read indexes with compressed fields, but as soon as merges
> >>> occur or the index is optimized, all compressed fields are decompressed and
> >>> converted to Field.Store.YES. Because of this, indexes with compressed
> >>> fields can suddenly get larger.
> >>>
> >>>
> >>>
> >>> While we generally try and maintain full backwards compatibility between
> >>> major versions, Lucene 3.0 has some minor breaks, mostly related to
> >>> deprecation removal, pointed out in the 'Changes in backwards compatibility
> >>> policy' section of CHANGES.txt. Notable are:
> >>>
> >>>
> >>>
> >>> - IndexReader.open(Directory) now opens in read-only mode per default (this
> >>> method was deprecated because of that in 2.9). The same occurs to
> >>> IndexSearcher.
> >>>
> >>> - Already started in 2.9, core TokenStreams are now made final to enforce
> >>> the decorator pattern.
> >>>
> >>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws an
> >>> unchecked ThreadInterruptedException that extends RuntimeException and
> >>> clears the interrupt status.
> >>>
> >>>
> >>>
> >>> Also, remember that this is a release candidate, and not the final Lucene
> >>> 3.0 release.
> >>>
> >>>
> >>>
> >>> You can find the full list of changes here:
> >>>
> >>>
> >>>
> >>> HTML version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> >>> hanges.html
> >>>
> >>>
> >>>
> >>> Text version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> >>> hanges.txt
> >>>
> >>>
> >>>
> >>> Changes have also occurred in Lucene's contrib area:
> >>>
> >>>
> >>>
> >>> HTML version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> >>> ontrib-Changes.html
> >>>
> >>>
> >>>
> >>> Text version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> >>> ontrib-Changes.txt
> >>>
> >>>
> >>>
> >>> Download release candidate 1 here:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
> >>>
> >>>
> >>>
> >>> Be sure to report back with any issues you find! Look especially for faults
> >>> in generification of public APIs (like missing wildcards,...).
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> Uwe Schindler
> >>>
> >>>
> >>>
> >>> -----
> >>>
> >>> Uwe Schindler
> >>>
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>>
> >>> http://www.thetaphi.de
> >>>
> >>> eMail: uwe@thetaphi.de
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >
> >
> > --
> > - Mark
> >
> > http://www.lucidimagination.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
>
> --
>
> -
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Glen Newton <gl...@gmail.com>.
I understand the reasons, but - if I may ask so late in the game - was
this the best way to do this?
>From a user (developer) perspective, this is an implementation issue.
Couldn't this have been done behind the scenes, so that when I asked
for Field.Index.ANALYZED && Field.Store.COMPRESS, instead of what
previously happened (and was variously problematic), two fields were
transparently created, one being binary compressed stored and the
other being indexed only? The Field API could hide all of this
complexity, using one underlying Field when I use Field.getString()
(compressed stored one), using the other when I use Field.setBoost()
(the indexed one) and both when I call Field.setValue(). This might
have less impact on developers and be less disruptive on API changes.
Oh, some naming convention could handle the underlying Fields.
A little complicated I agree.
Again, apologies to those who worked hard on these changes: my fault
for not noticing this sooner (I hadn't started moving my code to 2.9
from 2.4 so I hadn't read the deprecation signs).
thanks,
Glen
2009/11/17 Mark Miller <ma...@gmail.com>:
> Here is some of the history:
>
> https://issues.apache.org/jira/browse/LUCENE-652
> https://issues.apache.org/jira/browse/LUCENE-1960
>
> Glen Newton wrote:
>> Could someone send me where the rationale for the removal of
>> COMPRESSED fields is? I've looked at
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
>> but it is a little light on the 'why' of this change.
>>
>> My fault - of course - for not paying attention.
>>
>> thanks,
>> Glen
>>
>> 2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
>>
>>> Hello Lucene users,
>>>
>>>
>>>
>>> On behalf of the Lucene dev community (a growing community far larger than
>>> just the committers) I would like to announce the first release candidate
>>> for Lucene Java 3.0.
>>>
>>>
>>>
>>> Please download and check it out - take it for a spin and kick the tires. If
>>> all goes well, we hope to release the final version of Lucene 3.0 in a
>>> little over a week.
>>>
>>>
>>>
>>> The new version is mostly a cleanup release without any new features. All
>>> deprecations targeted to be removed in version 3.0 were removed. If you are
>>> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
>>> warnings in your code base to be able to recompile against this version.
>>>
>>>
>>>
>>> This is the first Lucene release with Java 5 as a minimum requirement. The
>>> API was cleaned up to make use of Java 5's generics, varargs, enums, and
>>> autoboxing. New users of Lucene are advised to use this version for new
>>> developments, because it has a clean, type safe new API. Upgrading users can
>>> now remove unnecessary casts and add generics to their code, too. If you
>>> have not upgraded your installation to Java 5, please read the file
>>> JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
>>> 3.0, it will also happen with any previous release when you upgrade your
>>> Java environment).
>>>
>>>
>>>
>>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
>>> deprecated compressed fields; support for them was removed now. Lucene 3.0
>>> is still able to read indexes with compressed fields, but as soon as merges
>>> occur or the index is optimized, all compressed fields are decompressed and
>>> converted to Field.Store.YES. Because of this, indexes with compressed
>>> fields can suddenly get larger.
>>>
>>>
>>>
>>> While we generally try and maintain full backwards compatibility between
>>> major versions, Lucene 3.0 has some minor breaks, mostly related to
>>> deprecation removal, pointed out in the 'Changes in backwards compatibility
>>> policy' section of CHANGES.txt. Notable are:
>>>
>>>
>>>
>>> - IndexReader.open(Directory) now opens in read-only mode per default (this
>>> method was deprecated because of that in 2.9). The same occurs to
>>> IndexSearcher.
>>>
>>> - Already started in 2.9, core TokenStreams are now made final to enforce
>>> the decorator pattern.
>>>
>>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws an
>>> unchecked ThreadInterruptedException that extends RuntimeException and
>>> clears the interrupt status.
>>>
>>>
>>>
>>> Also, remember that this is a release candidate, and not the final Lucene
>>> 3.0 release.
>>>
>>>
>>>
>>> You can find the full list of changes here:
>>>
>>>
>>>
>>> HTML version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>>> hanges.html
>>>
>>>
>>>
>>> Text version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>>> hanges.txt
>>>
>>>
>>>
>>> Changes have also occurred in Lucene's contrib area:
>>>
>>>
>>>
>>> HTML version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>>> ontrib-Changes.html
>>>
>>>
>>>
>>> Text version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>>> ontrib-Changes.txt
>>>
>>>
>>>
>>> Download release candidate 1 here:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
>>>
>>>
>>>
>>> Be sure to report back with any issues you find! Look especially for faults
>>> in generification of public APIs (like missing wildcards,...).
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Uwe Schindler
>>>
>>>
>>>
>>> -----
>>>
>>> Uwe Schindler
>>>
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>
>>> http://www.thetaphi.de
>>>
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
-
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Mark Miller <ma...@gmail.com>.
Here is some of the history:
https://issues.apache.org/jira/browse/LUCENE-652
https://issues.apache.org/jira/browse/LUCENE-1960
Glen Newton wrote:
> Could someone send me where the rationale for the removal of
> COMPRESSED fields is? I've looked at
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
> but it is a little light on the 'why' of this change.
>
> My fault - of course - for not paying attention.
>
> thanks,
> Glen
>
> 2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
>
>> Hello Lucene users,
>>
>>
>>
>> On behalf of the Lucene dev community (a growing community far larger than
>> just the committers) I would like to announce the first release candidate
>> for Lucene Java 3.0.
>>
>>
>>
>> Please download and check it out - take it for a spin and kick the tires. If
>> all goes well, we hope to release the final version of Lucene 3.0 in a
>> little over a week.
>>
>>
>>
>> The new version is mostly a cleanup release without any new features. All
>> deprecations targeted to be removed in version 3.0 were removed. If you are
>> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
>> warnings in your code base to be able to recompile against this version.
>>
>>
>>
>> This is the first Lucene release with Java 5 as a minimum requirement. The
>> API was cleaned up to make use of Java 5's generics, varargs, enums, and
>> autoboxing. New users of Lucene are advised to use this version for new
>> developments, because it has a clean, type safe new API. Upgrading users can
>> now remove unnecessary casts and add generics to their code, too. If you
>> have not upgraded your installation to Java 5, please read the file
>> JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
>> 3.0, it will also happen with any previous release when you upgrade your
>> Java environment).
>>
>>
>>
>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
>> deprecated compressed fields; support for them was removed now. Lucene 3.0
>> is still able to read indexes with compressed fields, but as soon as merges
>> occur or the index is optimized, all compressed fields are decompressed and
>> converted to Field.Store.YES. Because of this, indexes with compressed
>> fields can suddenly get larger.
>>
>>
>>
>> While we generally try and maintain full backwards compatibility between
>> major versions, Lucene 3.0 has some minor breaks, mostly related to
>> deprecation removal, pointed out in the 'Changes in backwards compatibility
>> policy' section of CHANGES.txt. Notable are:
>>
>>
>>
>> - IndexReader.open(Directory) now opens in read-only mode per default (this
>> method was deprecated because of that in 2.9). The same occurs to
>> IndexSearcher.
>>
>> - Already started in 2.9, core TokenStreams are now made final to enforce
>> the decorator pattern.
>>
>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws an
>> unchecked ThreadInterruptedException that extends RuntimeException and
>> clears the interrupt status.
>>
>>
>>
>> Also, remember that this is a release candidate, and not the final Lucene
>> 3.0 release.
>>
>>
>>
>> You can find the full list of changes here:
>>
>>
>>
>> HTML version:
>>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> hanges.html
>>
>>
>>
>> Text version:
>>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> hanges.txt
>>
>>
>>
>> Changes have also occurred in Lucene's contrib area:
>>
>>
>>
>> HTML version:
>>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> ontrib-Changes.html
>>
>>
>>
>> Text version:
>>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
>> ontrib-Changes.txt
>>
>>
>>
>> Download release candidate 1 here:
>>
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
>>
>>
>>
>> Be sure to report back with any issues you find! Look especially for faults
>> in generification of public APIs (like missing wildcards,...).
>>
>>
>>
>> Thanks,
>>
>> Uwe Schindler
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail: uwe@thetaphi.de
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Lucene Java 3.0.0 RC1 now available for testing
Posted by Uwe Schindler <uw...@thetaphi.de>.
Yes, just add the field two times to Document (with the same name), to
achieve this.
Using the same name is no problem, as between stored and inverted fields no
relation exists. Lucene always created internally "two fields" with the same
name. You can still do this, but if you want to compress, you have to do it
manually by creating the compressed byte[] and adding it as stored-only
binary field.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Chris Lu [mailto:chris.lu@gmail.com]
> Sent: Wednesday, November 18, 2009 1:06 AM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>
> So will I need to use 2 fields, one filed is analyzed and the other
> field is binary, to replace one compressed fields previously?
>
> --
> Chris Lu
> -------------------------
> Instant Scalable Full-Text Search On Any Database/Application
> site: http://www.dbsight.net
> demo: http://search.dbsight.com
> Lucene Database Search in 3 minutes:
> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3
> _minutes
> DBSight customer, a shopping comparison site, (anonymous per request) got
> 2.6 Million Euro funding!
>
>
>
> Uwe Schindler wrote:
> > Because you can do the compression yourself by just adding a binary
> stored
> > field with the compressed content. And then you can use any algorithm,
> even
> > bz2 or whatever.
> >
> > The problem is that the compressed fields made lot's of problems and
> special
> > cases during merging, because they were always decompressed,
> recompressed
> > and so on. If you need compressed fields, do it yourself, there is a
> new
> > class since 2.9.0 called CompressionUtils that can compress strings or
> > byte[] to compressed byte[]. Add the result to your index as binary
> stored
> > field and voila.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: Glen Newton [mailto:glen.newton@gmail.com]
> >> Sent: Tuesday, November 17, 2009 11:36 PM
> >> To: java-user@lucene.apache.org
> >> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
> >>
> >> Could someone send me where the rationale for the removal of
> >> COMPRESSED fields is? I've looked at
> >> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> >> rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
> >> but it is a little light on the 'why' of this change.
> >>
> >> My fault - of course - for not paying attention.
> >>
> >> thanks,
> >> Glen
> >>
> >> 2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
> >>
> >>> Hello Lucene users,
> >>>
> >>>
> >>>
> >>> On behalf of the Lucene dev community (a growing community far larger
> >>>
> >> than
> >>
> >>> just the committers) I would like to announce the first release
> >>>
> >> candidate
> >>
> >>> for Lucene Java 3.0.
> >>>
> >>>
> >>>
> >>> Please download and check it out - take it for a spin and kick the
> >>>
> >> tires. If
> >>
> >>> all goes well, we hope to release the final version of Lucene 3.0 in a
> >>> little over a week.
> >>>
> >>>
> >>>
> >>> The new version is mostly a cleanup release without any new features.
> >>>
> >> All
> >>
> >>> deprecations targeted to be removed in version 3.0 were removed. If
> you
> >>>
> >> are
> >>
> >>> upgrading from version 2.9.1 of Lucene, you have to fix all
> deprecation
> >>> warnings in your code base to be able to recompile against this
> version.
> >>>
> >>>
> >>>
> >>> This is the first Lucene release with Java 5 as a minimum requirement.
> >>>
> >> The
> >>
> >>> API was cleaned up to make use of Java 5's generics, varargs, enums,
> and
> >>> autoboxing. New users of Lucene are advised to use this version for
> new
> >>> developments, because it has a clean, type safe new API. Upgrading
> users
> >>>
> >> can
> >>
> >>> now remove unnecessary casts and add generics to their code, too. If
> you
> >>> have not upgraded your installation to Java 5, please read the file
> >>> JRE_VERSION_MIGRATION.txt (please note that this is not related to
> >>>
> >> Lucene
> >>
> >>> 3.0, it will also happen with any previous release when you upgrade
> your
> >>> Java environment).
> >>>
> >>>
> >>>
> >>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
> >>> deprecated compressed fields; support for them was removed now. Lucene
> >>>
> >> 3.0
> >>
> >>> is still able to read indexes with compressed fields, but as soon as
> >>>
> >> merges
> >>
> >>> occur or the index is optimized, all compressed fields are
> decompressed
> >>>
> >> and
> >>
> >>> converted to Field.Store.YES. Because of this, indexes with compressed
> >>> fields can suddenly get larger.
> >>>
> >>>
> >>>
> >>> While we generally try and maintain full backwards compatibility
> between
> >>> major versions, Lucene 3.0 has some minor breaks, mostly related to
> >>> deprecation removal, pointed out in the 'Changes in backwards
> >>>
> >> compatibility
> >>
> >>> policy' section of CHANGES.txt. Notable are:
> >>>
> >>>
> >>>
> >>> - IndexReader.open(Directory) now opens in read-only mode per default
> >>>
> >> (this
> >>
> >>> method was deprecated because of that in 2.9). The same occurs to
> >>> IndexSearcher.
> >>>
> >>> - Already started in 2.9, core TokenStreams are now made final to
> >>>
> >> enforce
> >>
> >>> the decorator pattern.
> >>>
> >>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws
> >>>
> >> an
> >>
> >>> unchecked ThreadInterruptedException that extends RuntimeException and
> >>> clears the interrupt status.
> >>>
> >>>
> >>>
> >>> Also, remember that this is a release candidate, and not the final
> >>>
> >> Lucene
> >>
> >>> 3.0 release.
> >>>
> >>>
> >>>
> >>> You can find the full list of changes here:
> >>>
> >>>
> >>>
> >>> HTML version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> >>>
> >> rc1/changes/C
> >>
> >>> hanges.html
> >>>
> >>>
> >>>
> >>> Text version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> >>>
> >> rc1/changes/C
> >>
> >>> hanges.txt
> >>>
> >>>
> >>>
> >>> Changes have also occurred in Lucene's contrib area:
> >>>
> >>>
> >>>
> >>> HTML version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> >>>
> >> rc1/changes/C
> >>
> >>> ontrib-Changes.html
> >>>
> >>>
> >>>
> >>> Text version:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> >>>
> >> rc1/changes/C
> >>
> >>> ontrib-Changes.txt
> >>>
> >>>
> >>>
> >>> Download release candidate 1 here:
> >>>
> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
> >>>
> >>>
> >>>
> >>> Be sure to report back with any issues you find! Look especially for
> >>>
> >> faults
> >>
> >>> in generification of public APIs (like missing wildcards,...).
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> Uwe Schindler
> >>>
> >>>
> >>>
> >>> -----
> >>>
> >>> Uwe Schindler
> >>>
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>>
> >>> http://www.thetaphi.de
> >>>
> >>> eMail: uwe@thetaphi.de
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >> --
> >>
> >> -
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Chris Lu <ch...@gmail.com>.
So will I need to use 2 fields, one filed is analyzed and the other
field is binary, to replace one compressed fields previously?
--
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes: http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per request) got 2.6 Million Euro funding!
Uwe Schindler wrote:
> Because you can do the compression yourself by just adding a binary stored
> field with the compressed content. And then you can use any algorithm, even
> bz2 or whatever.
>
> The problem is that the compressed fields made lot's of problems and special
> cases during merging, because they were always decompressed, recompressed
> and so on. If you need compressed fields, do it yourself, there is a new
> class since 2.9.0 called CompressionUtils that can compress strings or
> byte[] to compressed byte[]. Add the result to your index as binary stored
> field and voila.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Glen Newton [mailto:glen.newton@gmail.com]
>> Sent: Tuesday, November 17, 2009 11:36 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>>
>> Could someone send me where the rationale for the removal of
>> COMPRESSED fields is? I've looked at
>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
>> rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
>> but it is a little light on the 'why' of this change.
>>
>> My fault - of course - for not paying attention.
>>
>> thanks,
>> Glen
>>
>> 2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
>>
>>> Hello Lucene users,
>>>
>>>
>>>
>>> On behalf of the Lucene dev community (a growing community far larger
>>>
>> than
>>
>>> just the committers) I would like to announce the first release
>>>
>> candidate
>>
>>> for Lucene Java 3.0.
>>>
>>>
>>>
>>> Please download and check it out - take it for a spin and kick the
>>>
>> tires. If
>>
>>> all goes well, we hope to release the final version of Lucene 3.0 in a
>>> little over a week.
>>>
>>>
>>>
>>> The new version is mostly a cleanup release without any new features.
>>>
>> All
>>
>>> deprecations targeted to be removed in version 3.0 were removed. If you
>>>
>> are
>>
>>> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
>>> warnings in your code base to be able to recompile against this version.
>>>
>>>
>>>
>>> This is the first Lucene release with Java 5 as a minimum requirement.
>>>
>> The
>>
>>> API was cleaned up to make use of Java 5's generics, varargs, enums, and
>>> autoboxing. New users of Lucene are advised to use this version for new
>>> developments, because it has a clean, type safe new API. Upgrading users
>>>
>> can
>>
>>> now remove unnecessary casts and add generics to their code, too. If you
>>> have not upgraded your installation to Java 5, please read the file
>>> JRE_VERSION_MIGRATION.txt (please note that this is not related to
>>>
>> Lucene
>>
>>> 3.0, it will also happen with any previous release when you upgrade your
>>> Java environment).
>>>
>>>
>>>
>>> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
>>> deprecated compressed fields; support for them was removed now. Lucene
>>>
>> 3.0
>>
>>> is still able to read indexes with compressed fields, but as soon as
>>>
>> merges
>>
>>> occur or the index is optimized, all compressed fields are decompressed
>>>
>> and
>>
>>> converted to Field.Store.YES. Because of this, indexes with compressed
>>> fields can suddenly get larger.
>>>
>>>
>>>
>>> While we generally try and maintain full backwards compatibility between
>>> major versions, Lucene 3.0 has some minor breaks, mostly related to
>>> deprecation removal, pointed out in the 'Changes in backwards
>>>
>> compatibility
>>
>>> policy' section of CHANGES.txt. Notable are:
>>>
>>>
>>>
>>> - IndexReader.open(Directory) now opens in read-only mode per default
>>>
>> (this
>>
>>> method was deprecated because of that in 2.9). The same occurs to
>>> IndexSearcher.
>>>
>>> - Already started in 2.9, core TokenStreams are now made final to
>>>
>> enforce
>>
>>> the decorator pattern.
>>>
>>> - If you interrupt an IndexWriter merge thread, IndexWriter now throws
>>>
>> an
>>
>>> unchecked ThreadInterruptedException that extends RuntimeException and
>>> clears the interrupt status.
>>>
>>>
>>>
>>> Also, remember that this is a release candidate, and not the final
>>>
>> Lucene
>>
>>> 3.0 release.
>>>
>>>
>>>
>>> You can find the full list of changes here:
>>>
>>>
>>>
>>> HTML version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
>>>
>> rc1/changes/C
>>
>>> hanges.html
>>>
>>>
>>>
>>> Text version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
>>>
>> rc1/changes/C
>>
>>> hanges.txt
>>>
>>>
>>>
>>> Changes have also occurred in Lucene's contrib area:
>>>
>>>
>>>
>>> HTML version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
>>>
>> rc1/changes/C
>>
>>> ontrib-Changes.html
>>>
>>>
>>>
>>> Text version:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
>>>
>> rc1/changes/C
>>
>>> ontrib-Changes.txt
>>>
>>>
>>>
>>> Download release candidate 1 here:
>>>
>>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
>>>
>>>
>>>
>>> Be sure to report back with any issues you find! Look especially for
>>>
>> faults
>>
>>> in generification of public APIs (like missing wildcards,...).
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Uwe Schindler
>>>
>>>
>>>
>>> -----
>>>
>>> Uwe Schindler
>>>
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>
>>> http://www.thetaphi.de
>>>
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> --
>>
>> -
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
RE: Lucene Java 3.0.0 RC1 now available for testing
Posted by Uwe Schindler <uw...@thetaphi.de>.
Because you can do the compression yourself by just adding a binary stored
field with the compressed content. And then you can use any algorithm, even
bz2 or whatever.
The problem is that the compressed fields made lot's of problems and special
cases during merging, because they were always decompressed, recompressed
and so on. If you need compressed fields, do it yourself, there is a new
class since 2.9.0 called CompressionUtils that can compress strings or
byte[] to compressed byte[]. Add the result to your index as binary stored
field and voila.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Glen Newton [mailto:glen.newton@gmail.com]
> Sent: Tuesday, November 17, 2009 11:36 PM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing
>
> Could someone send me where the rationale for the removal of
> COMPRESSED fields is? I've looked at
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
> but it is a little light on the 'why' of this change.
>
> My fault - of course - for not paying attention.
>
> thanks,
> Glen
>
> 2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
> > Hello Lucene users,
> >
> >
> >
> > On behalf of the Lucene dev community (a growing community far larger
> than
> > just the committers) I would like to announce the first release
> candidate
> > for Lucene Java 3.0.
> >
> >
> >
> > Please download and check it out - take it for a spin and kick the
> tires. If
> > all goes well, we hope to release the final version of Lucene 3.0 in a
> > little over a week.
> >
> >
> >
> > The new version is mostly a cleanup release without any new features.
> All
> > deprecations targeted to be removed in version 3.0 were removed. If you
> are
> > upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
> > warnings in your code base to be able to recompile against this version.
> >
> >
> >
> > This is the first Lucene release with Java 5 as a minimum requirement.
> The
> > API was cleaned up to make use of Java 5's generics, varargs, enums, and
> > autoboxing. New users of Lucene are advised to use this version for new
> > developments, because it has a clean, type safe new API. Upgrading users
> can
> > now remove unnecessary casts and add generics to their code, too. If you
> > have not upgraded your installation to Java 5, please read the file
> > JRE_VERSION_MIGRATION.txt (please note that this is not related to
> Lucene
> > 3.0, it will also happen with any previous release when you upgrade your
> > Java environment).
> >
> >
> >
> > Lucene 3.0 has some changes regarding compressed fields: 2.9 already
> > deprecated compressed fields; support for them was removed now. Lucene
> 3.0
> > is still able to read indexes with compressed fields, but as soon as
> merges
> > occur or the index is optimized, all compressed fields are decompressed
> and
> > converted to Field.Store.YES. Because of this, indexes with compressed
> > fields can suddenly get larger.
> >
> >
> >
> > While we generally try and maintain full backwards compatibility between
> > major versions, Lucene 3.0 has some minor breaks, mostly related to
> > deprecation removal, pointed out in the 'Changes in backwards
> compatibility
> > policy' section of CHANGES.txt. Notable are:
> >
> >
> >
> > - IndexReader.open(Directory) now opens in read-only mode per default
> (this
> > method was deprecated because of that in 2.9). The same occurs to
> > IndexSearcher.
> >
> > - Already started in 2.9, core TokenStreams are now made final to
> enforce
> > the decorator pattern.
> >
> > - If you interrupt an IndexWriter merge thread, IndexWriter now throws
> an
> > unchecked ThreadInterruptedException that extends RuntimeException and
> > clears the interrupt status.
> >
> >
> >
> > Also, remember that this is a release candidate, and not the final
> Lucene
> > 3.0 release.
> >
> >
> >
> > You can find the full list of changes here:
> >
> >
> >
> > HTML version:
> >
> > http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> > hanges.html
> >
> >
> >
> > Text version:
> >
> > http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> > hanges.txt
> >
> >
> >
> > Changes have also occurred in Lucene's contrib area:
> >
> >
> >
> > HTML version:
> >
> > http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> > ontrib-Changes.html
> >
> >
> >
> > Text version:
> >
> > http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-
> rc1/changes/C
> > ontrib-Changes.txt
> >
> >
> >
> > Download release candidate 1 here:
> >
> > http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
> >
> >
> >
> > Be sure to report back with any issues you find! Look especially for
> faults
> > in generification of public APIs (like missing wildcards,...).
> >
> >
> >
> > Thanks,
> >
> > Uwe Schindler
> >
> >
> >
> > -----
> >
> > Uwe Schindler
> >
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> >
> > http://www.thetaphi.de
> >
> > eMail: uwe@thetaphi.de
> >
> >
> >
> >
> >
> >
>
>
>
> --
>
> -
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Java 3.0.0 RC1 now available for testing
Posted by Glen Newton <gl...@gmail.com>.
Could someone send me where the rationale for the removal of
COMPRESSED fields is? I've looked at
http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior
but it is a little light on the 'why' of this change.
My fault - of course - for not paying attention.
thanks,
Glen
2009/11/17 Uwe Schindler <uw...@thetaphi.de>:
> Hello Lucene users,
>
>
>
> On behalf of the Lucene dev community (a growing community far larger than
> just the committers) I would like to announce the first release candidate
> for Lucene Java 3.0.
>
>
>
> Please download and check it out - take it for a spin and kick the tires. If
> all goes well, we hope to release the final version of Lucene 3.0 in a
> little over a week.
>
>
>
> The new version is mostly a cleanup release without any new features. All
> deprecations targeted to be removed in version 3.0 were removed. If you are
> upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
> warnings in your code base to be able to recompile against this version.
>
>
>
> This is the first Lucene release with Java 5 as a minimum requirement. The
> API was cleaned up to make use of Java 5's generics, varargs, enums, and
> autoboxing. New users of Lucene are advised to use this version for new
> developments, because it has a clean, type safe new API. Upgrading users can
> now remove unnecessary casts and add generics to their code, too. If you
> have not upgraded your installation to Java 5, please read the file
> JRE_VERSION_MIGRATION.txt (please note that this is not related to Lucene
> 3.0, it will also happen with any previous release when you upgrade your
> Java environment).
>
>
>
> Lucene 3.0 has some changes regarding compressed fields: 2.9 already
> deprecated compressed fields; support for them was removed now. Lucene 3.0
> is still able to read indexes with compressed fields, but as soon as merges
> occur or the index is optimized, all compressed fields are decompressed and
> converted to Field.Store.YES. Because of this, indexes with compressed
> fields can suddenly get larger.
>
>
>
> While we generally try and maintain full backwards compatibility between
> major versions, Lucene 3.0 has some minor breaks, mostly related to
> deprecation removal, pointed out in the 'Changes in backwards compatibility
> policy' section of CHANGES.txt. Notable are:
>
>
>
> - IndexReader.open(Directory) now opens in read-only mode per default (this
> method was deprecated because of that in 2.9). The same occurs to
> IndexSearcher.
>
> - Already started in 2.9, core TokenStreams are now made final to enforce
> the decorator pattern.
>
> - If you interrupt an IndexWriter merge thread, IndexWriter now throws an
> unchecked ThreadInterruptedException that extends RuntimeException and
> clears the interrupt status.
>
>
>
> Also, remember that this is a release candidate, and not the final Lucene
> 3.0 release.
>
>
>
> You can find the full list of changes here:
>
>
>
> HTML version:
>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> hanges.html
>
>
>
> Text version:
>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> hanges.txt
>
>
>
> Changes have also occurred in Lucene's contrib area:
>
>
>
> HTML version:
>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> ontrib-Changes.html
>
>
>
> Text version:
>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/changes/C
> ontrib-Changes.txt
>
>
>
> Download release candidate 1 here:
>
> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/
>
>
>
> Be sure to report back with any issues you find! Look especially for faults
> in generification of public APIs (like missing wildcards,...).
>
>
>
> Thanks,
>
> Uwe Schindler
>
>
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: uwe@thetaphi.de
>
>
>
>
>
>
--
-
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org