Suggester using up memory

[Nick Vercammen <ni...@zeticon.com>]

Hey,

We have a problem on one of our installations with the suggestComponent.

The index has about 16 million documents and contains a "Global" field
which contains the data of multiple other fields. This "Global" field is
used to build up the suggestions.

A short time after starting Solr it is killed by oom_solr as it has run out
of memory. We configured solr to use 8GB max for the heap, which is a lot
more than what we use in other installations.

Configuration of the suggestComponent:

<searchComponent name="suggest" class="solr.SuggestComponent">
    <lst name="suggester">
        <str name="name">default</str>
        <str name="field">Global</str>
        <str name="lookupImpl">FreeTextLookupFactory</str>
        <str name="storeDir">suggester_fuzzy_dir</str>
        <str name="dictionaryImpl">HighFrequencyDictionaryFactory</str>
        <str name="suggestFreeTextAnalyzerFieldType">TextField</str>
        <str name="suggestAnalyzerFieldType">TextField</str>
        <str name="buildOnCommit">false</str>
        <str name="buildOnOptimize">false</str>
    </lst>
</searchComponent>

Is there a problem in our configuration?How can we make sure the build
up of the suggester does not run out of memory? How do we determine
the actual memory requirements in this case?

Thanks in advance!


Nick