You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jim Ferenczi (JIRA)" <ji...@apache.org> on 2018/12/03 10:27:00 UTC
[jira] [Created] (LUCENE-8584) Japanese UserDictionary should
remove duplicate entries
Jim Ferenczi created LUCENE-8584:
------------------------------------
Summary: Japanese UserDictionary should remove duplicate entries
Key: LUCENE-8584
URL: https://issues.apache.org/jira/browse/LUCENE-8584
Project: Lucene - Core
Issue Type: Improvement
Reporter: Jim Ferenczi
The Japanese's UserDictionary in the kuromoji module fails to load the dictionary if the file contains duplicate entries:
{noformat}
java.lang.UnsupportedOperationException
at __randomizedtesting.SeedInfo.seed([C340BE6DB5DF33E8:A804576E05DF86DF]:0)
at org.apache.lucene.util.fst.Outputs.merge(Outputs.java:97)
at org.apache.lucene.util.fst.Builder.add(Builder.java:445)
at org.apache.lucene.analysis.ja.dict.UserDictionary.<init>(UserDictionary.java:135)
at org.apache.lucene.analysis.ja.dict.UserDictionary.open(UserDictionary.java:81)
at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.readDict(TestJapaneseTokenizer.java:55)
at org.apache.lucene.analysis.ja.dict.UserDictionaryTest.testLookup(UserDictionaryTest.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
Duplicate entries should be ignored or a more descriptive error should be thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org