TokenNameFinderTrainer not working

[Raimon Bosch <ra...@gmail.com>]

Hi all,

I'm trying the TokenNameFinderTraining but I'm getting some mistakes. I
think that the system is not enough flexible reading files.

bin/opennlp TokenNameFinderTrainer -encoding UTF-8 -lang es -data
/tmp/es-ner-artists.train -model models/es-ner-artist.bin
Indexing events using cutoff of 5

Computing event counts...  java.io.IOException: Found unexpected annotation
<END>.
Incorporating indexed data for training...
Exception in thread "main" java.lang.NullPointerException
at opennlp.maxent.GISTrainer.trainModel(GISTrainer.java:272)
at opennlp.maxent.GISTrainer.trainModel(GISTrainer.java:252)
at opennlp.maxent.GIS.trainModel(GIS.java:228)
at opennlp.maxent.GIS.trainModel(GIS.java:179)
at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:345)
at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:356)
at
opennlp.tools.cmdline.namefind.TokenNameFinderTrainerTool.run(TokenNameFinderTrainerTool.java:87)
at opennlp.tools.cmdline.CLI.main(CLI.java:183)

I''ve attached the file 'es-ner-artists.train'

Regards,
Raimon Bosch.

Re: TokenNameFinderTrainer not working

[Jörn Kottmann <ko...@gmail.com>]

That is a mistake in your training data, at some place your start tag is 
not terminated
by an end tag. All tokens must be white space terminated, maybe your end 
tag is not delimited
by a white space which can be hard to see.

We know that the exception you are getting makes it hard to find. In the 
next version
the exception will contain a little more context.

Jörn

On 7/29/11 5:29 PM, Raimon Bosch wrote:
>
> Hi all,
>
> I'm trying the TokenNameFinderTraining but I'm getting some mistakes. 
> I think that the system is not enough flexible reading files.
>
> bin/opennlp TokenNameFinderTrainer -encoding UTF-8 -lang es -data 
> /tmp/es-ner-artists.train -model models/es-ner-artist.bin
> Indexing events using cutoff of 5
>
> Computing event counts...  java.io.IOException: Found unexpected 
> annotation <END>.
> Incorporating indexed data for training...
> Exception in thread "main" java.lang.NullPointerException
> at opennlp.maxent.GISTrainer.trainModel(GISTrainer.java:272)
> at opennlp.maxent.GISTrainer.trainModel(GISTrainer.java:252)
> at opennlp.maxent.GIS.trainModel(GIS.java:228)
> at opennlp.maxent.GIS.trainModel(GIS.java:179)
> at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:345)
> at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:356)
> at 
> opennlp.tools.cmdline.namefind.TokenNameFinderTrainerTool.run(TokenNameFinderTrainerTool.java:87)
> at opennlp.tools.cmdline.CLI.main(CLI.java:183)
>
> I''ve attached the file 'es-ner-artists.train'
>
> Regards,
> Raimon Bosch.