Re-created fields consistently indexed?

[Jason Polites <ja...@gmail.com>]

Hi all,

I understand that it is possible to "re-create" fields which are indexed but
not stored (as is done by Luke), and that this is a lossy process, however I
am wondering whether the indexed version of this remains consistent.

That is, if I re-create a non-stored field, then re-index this field in a
new document, will the indexed (inverted) data be the same as in the
original version?

Thanks.

Re: Re-created fields consistently indexed?

[Erik Hatcher <er...@ehatchersolutions.com>]

On Aug 30, 2006, at 11:07 AM, Jason Polites wrote:
> I understand that it is possible to "re-create" fields which are  
> indexed but
> not stored (as is done by Luke), and that this is a lossy process,  
> however I
> am wondering whether the indexed version of this remains consistent.
>
> That is, if I re-create a non-stored field, then re-index this  
> field in a
> new document, will the indexed (inverted) data be the same as in the
> original version?

That really all depends.  For example, the original analysis process  
could put in position increment gaps between sentences or overlay  
terms in same positions, and don't forget about term offsets.   Just  
pulling out the terms without getting those details would be even  
lossier.  I'm not even sure how you could recreate that even if you  
had all of that position and offset information without some kind of  
special Analyzer that would set things exactly as they were.  I  
suspect there is some low-level tricks that could be played to do  
this bypassing an analyzer.

	Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org