SpellCheckComponent buildOnOptimize

[Ondrej Kosarko <ko...@ufal.mff.cuni.cz>]

Hello everyone, 

I was wondering how does one use buildOnOptimize=true on SpellCheckComponent? 

From [ https://github.com/apache/solr/blob/516180f2ad2393038601f737319e043ec9ac923c/solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java#L824-L830 | https://github.com/apache/solr/blob/516180f2ad2393038601f737319e043ec9ac923c/solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java#L824-L830 ] it seems that the check is on segments count (if there's only one) and not on running the optimization (no idea if that's actually possible). 

For a moment I though sending <optimize maxSegments="1"/> to the update handler should do the trick; but then the docs at [ https://solr.apache.org/guide/solr/latest/indexing-guide/indexing-with-update-handlers.html | https://solr.apache.org/guide/solr/latest/indexing-guide/indexing-with-update-handlers.html  ] says: 

"Makes a best effort attempt to merge the segments down to no more than this number of segments but does not guarantee that the goal will be achieved." 




So how's that supposed to work? 




I was investigating the info messages "o.a.s.h.c.SpellCheckComponent Index is not optimized therefore skipping building spell check index for: ..." and why they don't disappear even after running optimize from within solr admin... 




Thanks for any clarifications. 

Best regards, 

Ondrej Kosarko 

Re: SpellCheckComponent buildOnOptimize

[Mikhail Khludnev <mk...@apache.org>]

Hello, Ondrej.
Right it seems like a trap. However,
https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java#L2079
gives a clue that optimize can end up with more than sigle segment only
when concurrent updates occurs. Presumably, you can pause updates in the
app, and hope that optimize kicks off spell check rebuild. Although it
looks like a band aid.

On Mon, Dec 19, 2022 at 6:10 PM Ondrej Kosarko <ko...@ufal.mff.cuni.cz>
wrote:

>
>
> Hello everyone,
>
> I was wondering how does one use buildOnOptimize=true on
> SpellCheckComponent?
>
> From [
> https://github.com/apache/solr/blob/516180f2ad2393038601f737319e043ec9ac923c/solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java#L824-L830
> |
> https://github.com/apache/solr/blob/516180f2ad2393038601f737319e043ec9ac923c/solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java#L824-L830
> ] it seems that the check is on segments count (if there's only one) and
> not on running the optimization (no idea if that's actually possible).
>
> For a moment I though sending <optimize maxSegments="1"/> to the update
> handler should do the trick; but then the docs at [
> https://solr.apache.org/guide/solr/latest/indexing-guide/indexing-with-update-handlers.html
> |
> https://solr.apache.org/guide/solr/latest/indexing-guide/indexing-with-update-handlers.html
> ] says:
>
> "Makes a best effort attempt to merge the segments down to no more than
> this number of segments but does not guarantee that the goal will be
> achieved."
>
>
>
>
> So how's that supposed to work?
>
>
>
>
> I was investigating the info messages "o.a.s.h.c.SpellCheckComponent Index
> is not optimized therefore skipping building spell check index for: ..."
> and why they don't disappear even after running optimize from within solr
> admin...
>
>
>
>
> Thanks for any clarifications.
>
> Best regards,
>
> Ondrej Kosarko
>
>
>

-- 
Sincerely yours
Mikhail Khludnev