How to get rid of unused fields?

["Khin, Gerald" <Ge...@ids-scheer.com>]

Hi folks!

 

After deleting documents from the index it can happen that fields become
unused (i.e. no document has this field anymore). And
IndexReader.getFieldNames() still returns these unused fields, even
after optimizing the index. Is there any chance to get rid of these
unused fields?

 

Cheers,

Gerald

 

RE: How to get rid of unused fields?

[Chris Hostetter <ho...@fucit.org>]

: : The same here, even with trunk from yesterday. If you create a field, it
: : stays there forever, even after deleting *all* documents from index,
: : reindexing without the field and optimizing.
: 
: Uwe: if you have a quick test case already written can you try it against 
: 2.4 (and maybe 2.3) because i'm pretty sure that use to work.

i just tested 2.4, 2.3, and 2.2 .. sure enough you can't seem to get rid 
of fields even when you delete all docs.

this seems like a bad thing.  so i'm oging to open an issue and hope that 
the big brains of Index Internals can see an easy way to make this work.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

RE: How to get rid of unused fields?

[Chris Hostetter <ho...@fucit.org>]

: The same here, even with trunk from yesterday. If you create a field, it
: stays there forever, even after deleting *all* documents from index,
: reindexing without the field and optimizing.

Uwe: if you have a quick test case already written can you try it against 
2.4 (and maybe 2.3) because i'm pretty sure that use to work.

(the reason i'm confident is because once upon a time, before solr 
implemented special logic to make "<delete><query>*:*</query></delete>" 
explicitly removing the index directory the FAQ on changing schemas told 
people to make sure to optimize before reindexing so the field stats would 
be removed.)


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

RE: How to get rid of unused fields?

[Uwe Schindler <uw...@thetaphi.de>]

The same here, even with trunk from yesterday. If you create a field, it
stays there forever, even after deleting *all* documents from index,
reindexing without the field and optimizing.

Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> Sent: Friday, July 17, 2009 12:39 AM
> To: java-user@lucene.apache.org
> Subject: Re: How to get rid of unused fields?
> 
> 
> : After deleting documents from the index it can happen that fields become
> : unused (i.e. no document has this field anymore). And
> : IndexReader.getFieldNames() still returns these unused fields, even
> : after optimizing the index. Is there any chance to get rid of these
> : unused fields?
> 
> that's odd ... i'm 99% sure that should work (optimizng should perge
> unused fields).
> 
> what version of Lucene are you using?  (if you're on the trunk it might be
> a bug introduced recently with some of the indexing changes)
> 
> Can you post some test code demonstrating the problem?
> 
> 
> 
> -Hoss
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: How to get rid of unused fields?

[Chris Hostetter <ho...@fucit.org>]

: After deleting documents from the index it can happen that fields become
: unused (i.e. no document has this field anymore). And
: IndexReader.getFieldNames() still returns these unused fields, even
: after optimizing the index. Is there any chance to get rid of these
: unused fields?

that's odd ... i'm 99% sure that should work (optimizng should perge 
unused fields).  

what version of Lucene are you using?  (if you're on the trunk it might be 
a bug introduced recently with some of the indexing changes)

Can you post some test code demonstrating the problem?



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org