You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Maciej Lisiewski (Updated) (JIRA)" <ji...@apache.org> on 2011/12/14 03:56:31 UTC
[jira] [Updated] (SOLR-2968) Hunspell very high memory use when
loading dictionary
[ https://issues.apache.org/jira/browse/SOLR-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maciej Lisiewski updated SOLR-2968:
-----------------------------------
Description:
Hunspell stemmer requires gigantic (for the task) amounts of memory to load dictionary/rules files.
For example loading a 4.5 MB polish dictionary (with empty index!) will cause whole core to crash with various out of memory errors unless you set max heap size close to 2GB or more.
By comparison Stempel using the same dictionary file works just fine with 1/8 of that (and possibly lower values as well).
Sample error log entries:
http://pastebin.com/fSrdd5W1
http://pastebin.com/Lmi0re7Z
was:
Hunspell stemmer requires gigantic (for the task) amounts of memory to load dictionary/rules files.
For example loading a 4.5 MB polish dictionary (with empty index!) will cause whole core to crash with various out of memory errors unless you set max heap size close to 2GB or more.
By comparison Stempel using the same dictionary file works just fine with 1/8 of that (and possibly lower values as well).
> Hunspell very high memory use when loading dictionary
> -----------------------------------------------------
>
> Key: SOLR-2968
> URL: https://issues.apache.org/jira/browse/SOLR-2968
> Project: Solr
> Issue Type: Bug
> Affects Versions: 3.5
> Reporter: Maciej Lisiewski
> Priority: Minor
>
> Hunspell stemmer requires gigantic (for the task) amounts of memory to load dictionary/rules files.
> For example loading a 4.5 MB polish dictionary (with empty index!) will cause whole core to crash with various out of memory errors unless you set max heap size close to 2GB or more.
> By comparison Stempel using the same dictionary file works just fine with 1/8 of that (and possibly lower values as well).
> Sample error log entries:
> http://pastebin.com/fSrdd5W1
> http://pastebin.com/Lmi0re7Z
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org