[jira] [Resolved] (LUCENE-8584) Japanese UserDictionary should remove duplicate entries

["Jim Ferenczi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jim Ferenczi resolved LUCENE-8584.
----------------------------------
    Resolution: Won't Fix

Fair enough. Per [~rcmuir]'s comment: duplicated rules should be detected on the app side.

> Japanese UserDictionary should remove duplicate entries
> -------------------------------------------------------
>
>                 Key: LUCENE-8584
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8584
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Jim Ferenczi
>            Priority: Minor
>         Attachments: LUCENE-8584.patch
>
>
> The Japanese's UserDictionary in the kuromoji module fails to load the dictionary if the file contains duplicate entries:
> {noformat}
> java.lang.UnsupportedOperationException
> 	at __randomizedtesting.SeedInfo.seed([C340BE6DB5DF33E8:A804576E05DF86DF]:0)
> 	at org.apache.lucene.util.fst.Outputs.merge(Outputs.java:97)
> 	at org.apache.lucene.util.fst.Builder.add(Builder.java:445)
> 	at org.apache.lucene.analysis.ja.dict.UserDictionary.<init>(UserDictionary.java:135)
> 	at org.apache.lucene.analysis.ja.dict.UserDictionary.open(UserDictionary.java:81)
> 	at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.readDict(TestJapaneseTokenizer.java:55)
> 	at org.apache.lucene.analysis.ja.dict.UserDictionaryTest.testLookup(UserDictionaryTest.java:30)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
> Duplicate entries should be ignored or a more descriptive error should be thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org