Loading a trained model

[Jeffrey Zemerick <jz...@apache.org>]

I successfully trained a model and ran the evaluator tool. I then tried to
load my model using the sample code in the documentation [1] that works
with the pre-built models available but no entities were ever identified. I
looked through the code of the Evaluator tool and found the model was being
loaded differently (via the TokenNameFinderModelLoader class) so I switched
to using that code and now my model works. I just wanted to see if there
might be any reason why the sample code in the documentation did not work.
I'm still new to this stuff but making progress!

Thanks,
Jeff

[1]
http://opennlp.apache.org/documentation/1.5.3/manual/opennlp.html#tools.namefind.recognition.api

Re: Loading a trained model

[Jeffrey Zemerick <jz...@apache.org>]

Thanks. I will check that out.

Jeff


On Tue, Oct 22, 2013 at 5:00 AM, Jörn Kottmann <ko...@gmail.com> wrote:

> On 10/21/2013 10:21 PM, Jeffrey Zemerick wrote:
>
>> And this worked. I was just interested in why the second code worked and
>> the first did not. I feel sure I had the path to the model correct in the
>> first code because it showed a debug line about loading the model and
>> delayed about 60 seconds. I did not see any errors - just an empty array
>> of
>> name spans.
>>
>
> The model itself is just a container of all the resources the NameFinderME
> needs. To detect
>  names, you need to instantiate the NameFinderME and feed it with
> sentences.
>
> The documentation explains this, have a look here:
> http://opennlp.apache.org/**documentation/1.5.3/manual/**
> opennlp.html#tools.namefind.**recognition.api<http://opennlp.apache.org/documentation/1.5.3/manual/opennlp.html#tools.namefind.recognition.api>
>
> Also have a look at the JavaDocs of the NameFinderME.
>
> The sentences don't need to be read from the command line, the samples
> just do this
> for illustrations purposes. In a production system you probably have some
> kind of data base
> which contains everything,
>
> HTH,
> Jörn
>

Re: Loading a trained model

[Jörn Kottmann <ko...@gmail.com>]

On 10/21/2013 10:21 PM, Jeffrey Zemerick wrote:
> And this worked. I was just interested in why the second code worked and
> the first did not. I feel sure I had the path to the model correct in the
> first code because it showed a debug line about loading the model and
> delayed about 60 seconds. I did not see any errors - just an empty array of
> name spans.

The model itself is just a container of all the resources the 
NameFinderME needs. To detect
  names, you need to instantiate the NameFinderME and feed it with 
sentences.

The documentation explains this, have a look here:
http://opennlp.apache.org/documentation/1.5.3/manual/opennlp.html#tools.namefind.recognition.api

Also have a look at the JavaDocs of the NameFinderME.

The sentences don't need to be read from the command line, the samples 
just do this
for illustrations purposes. In a production system you probably have 
some kind of data base
which contains everything,

HTH,
Jörn

Re: Loading a trained model

[Jeffrey Zemerick <jz...@apache.org>]

Yes, first I tried the sample code but with my model file:

InputStream modelIn = new FileInputStream("en-ner-person.bin");

try {
  TokenNameFinderModel model = new TokenNameFinderModel(modelIn);
  NameFinderME nameFinder = new NameFinderME(model);

      Span nameSpans[] = nameFinder.find(sentence);
      logger.debug(nameSpans.size());

      // do something with the names

    nameFinder.clearAdaptiveData()

}
catch (IOException e) {
  e.printStackTrace();
}
finally {
  if (modelIn != null) {
    try {
      modelIn.close();
    }
    catch (IOException e) {
    }
  }
}

When I ran it, it would load the model in about 60 seconds but never
identify any entities. I gave the same input to the evaluator tool and it
identified the identities. So then I changed to this code where sentence is
the input:

TokenNameFinderModel personModel = new
TokenNameFinderModelLoader().load(new File(personModelPath));
ObjectStream<String> untokenizedLineStream = new PlainTextByLineStream(new
StringReader(sentence));
NameFinderME nameFinder = new NameFinderME(model);
String line = untokenizedLineStream.read();
String whitespaceTokenizerLine[] =
WhitespaceTokenizer.INSTANCE.tokenize(line);
Span[] spans = nameFinder.find(whitespaceTokenizerLine);

And this worked. I was just interested in why the second code worked and
the first did not. I feel sure I had the path to the model correct in the
first code because it showed a debug line about loading the model and
delayed about 60 seconds. I did not see any errors - just an empty array of
name spans.

Thanks,
Jeff




On Mon, Oct 21, 2013 at 11:48 AM, Jörn Kottmann <ko...@gmail.com> wrote:

> On 10/21/2013 05:01 PM, Jeffrey Zemerick wrote:
>
>> I successfully trained a model and ran the evaluator tool. I then tried to
>> load my model using the sample code in the documentation [1] that works
>> with the pre-built models available but no entities were ever identified.
>> I
>> looked through the code of the Evaluator tool and found the model was
>> being
>> loaded differently (via the TokenNameFinderModelLoader class) so I
>> switched
>> to using that code and now my model works. I just wanted to see if there
>> might be any reason why the sample code in the documentation did not work.
>>
>
> The sample code should work. Can you share the code you used?
>
> Thanks,
> Jörn
>

Re: Loading a trained model

[Jörn Kottmann <ko...@gmail.com>]

On 10/21/2013 05:01 PM, Jeffrey Zemerick wrote:
> I successfully trained a model and ran the evaluator tool. I then tried to
> load my model using the sample code in the documentation [1] that works
> with the pre-built models available but no entities were ever identified. I
> looked through the code of the Evaluator tool and found the model was being
> loaded differently (via the TokenNameFinderModelLoader class) so I switched
> to using that code and now my model works. I just wanted to see if there
> might be any reason why the sample code in the documentation did not work.

The sample code should work. Can you share the code you used?

Thanks,
Jörn