You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Alan Woodward (Jira)" <ji...@apache.org> on 2021/04/29 08:12:00 UTC
[jira] [Resolved] (LUCENE-9930) UkrainianMorfologikAnalyzer reloads
its Dictionary for every new TokenStreamComponents instance
[ https://issues.apache.org/jira/browse/LUCENE-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Woodward resolved LUCENE-9930.
-----------------------------------
Fix Version/s: main (9.0)
Resolution: Fixed
> UkrainianMorfologikAnalyzer reloads its Dictionary for every new TokenStreamComponents instance
> -----------------------------------------------------------------------------------------------
>
> Key: LUCENE-9930
> URL: https://issues.apache.org/jira/browse/LUCENE-9930
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Alan Woodward
> Assignee: Alan Woodward
> Priority: Major
> Fix For: main (9.0)
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Large static data structures should be loaded in Analyzer constructors and shared between threads, but the UkrainianMorfologikAnalyzer is loading its dictionary in `createComponents`, which means it is reloaded and stored on every new analysis thread. If you have a large dictionary and highly concurrent indexing then this can lead to you running out of memory as multiple copies of the dictionary are held in thread locals.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org