Wither field compresed="true" ?

["David Smiley (@MITRE.org)" <DS...@mitre.org>]

I just noticed that field compression (e.g. compressed="true") is no longer
in Solr, nor can I find why this was done.  Can a committer offer an
explanation?  If the reason is that it eats up CPU, then I'd rather accept
this tradeoff for a big-data yet small query volume use case.

~ David Smiley

-----
 Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Wither-field-compresed-true-tp926288p926288.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Wither field compresed="true" ?

[Chris Hostetter <ho...@fucit.org>]

: > I just noticed that field compression (e.g. compressed="true") is no longer
: > in Solr, nor can I find why this was done.  Can a committer offer an

This is clearly spelled out in the "Upgrading from Solr 1.4" section of 
CHANGES.txt on trunk and branch 3x...

* Field compression is no longer supported. Fields that were formerly
  compressed will be uncompressed as index segments are merged. For
  shorter fields, this may actually be an improvement, as the compression
  used was not very good for short text. Some indexes may get larger though.

...allthough it definitely seems bad that we do't have an issue link 
explaining in more detail.

can someone familiar with this update CHANGES.txt to refer to the jira 
issue with more background?  I can only find LUCENE-1960 off the cuf 
right now -- but it's just where COMPRESS is removed, it has no refrence 
to where it was deprecated.




-Hoss

Re: Wither field compresed="true" ?

[Mark Miller <ma...@gmail.com>]

On 6/27/10 4:51 PM, David Smiley (@MITRE.org) wrote:
> 
> I just noticed that field compression (e.g. compressed="true") is no longer
> in Solr, nor can I find why this was done.  Can a committer offer an
> explanation?  If the reason is that it eats up CPU, then I'd rather accept
> this tradeoff for a big-data yet small query volume use case.
> 
> ~ David Smiley
> 
> -----
>  Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book

Lucene pulled compression support in a way that Solr had to break back
compatibility here.

The argument in Lucene's case was that you can do compression external
to Lucene and have more options in what you do. It was said that the
current compression was terrible for anything but very large fields. It
could actually be detrimental if used on shorter fields.

Solr could still do compression itself in the future - because it could
not really stay back compat though (Lucene will uncompress compressed
fields as it merges the index), we have punted on the feature for now.

-- 
- Mark

http://www.lucidimagination.com

Re: Wither field compresed="true" ?

[MitchK <mi...@web.de>]

David,

well, I am no committer, but I noticed that Lucene will no longer care of
compressing (I think this was because of the trouble when doing this) and
maybe this is the reason why Solr keeps this option no longer available.

Unfortunately, I do not have got any link for it, but I think this was said
in some changes.txt (at Nutch, I think).

Kind regards
- Mitch
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Wither-field-compresed-true-tp926288p929985.html
Sent from the Solr - User mailing list archive at Nabble.com.