[jira] [Closed] (OPENNLP-1166) TwoPassDataIndexer fails if features contain \n

["Suneel Marthi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OPENNLP-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suneel Marthi closed OPENNLP-1166.
----------------------------------

> TwoPassDataIndexer fails if features contain \n
> -----------------------------------------------
>
>                 Key: OPENNLP-1166
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1166
>             Project: OpenNLP
>          Issue Type: Improvement
>          Components: Machine Learning
>    Affects Versions: 1.8.3
>            Reporter: Peter Thygesen
>            Assignee: Peter Thygesen
>             Fix For: 1.8.4
>
>
> Training a model with Newline tokens causes TwoPassDataIndexer to throw exception
> Exception in thread "main" java.util.NoSuchElementException
>     at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
>     at opennlp.tools.ml.model.FileEventStream.read(FileEventStream.java:71)
>     at opennlp.tools.ml.model.FileEventStream.read(FileEventStream.java:35)
>     at opennlp.tools.ml.model.AbstractDataIndexer.index(AbstractDataIndexer.java:168)
>     at opennlp.tools.ml.model.TwoPassDataIndexer.index(TwoPassDataIndexer.java:72)
>     at opennlp.tools.ml.AbstractEventTrainer.getDataIndexer(AbstractEventTrainer.java:68)
>     at opennlp.tools.ml.AbstractEventTrainer.train(AbstractEventTrainer.java:90)
>     at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:244)
>     at opennlp.tools.cmdline.namefind.TokenNameFinderTrainerTool.run(TokenNameFinderTrainerTool.java:169)
>     at opennlp.tools.cmdline.CLI.main(CLI.java:256)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)