You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@opennlp.apache.org by "Jim - FooBar();" <ji...@gmail.com> on 2012/03/13 17:34:17 UTC
DictionaryNameFinder cross-validation?
Hey guys,
First of all i can imagine you must be sick and tired of me reporting a
bug or improvement every single day!That is the nature of open-source
though isn't it? :-)
Today's issue came literally out of nowhere! Again it has to do with
cross-validation but with the DictionaryNameFinder this time - NOT the
maxent model...Ok, here goes:
Basically, both the maxentNameFinder and the DictionaryNameFinder can do
NER. Also both classes implement TokenNameFinder so from Java's
perspective either can be passed as argument to the
TokenNameFinderCrossValidator constructor...However, i tried doing that
this morning, in an effort to get some numbers my dictionary, and all i
get is 0 precision, 0 recall and -1 FMeasure, regardless of finding
loads of drugs! I think (not 100% sure) the problem is that the
CrossValidator expects annotated text (in order to verify) but the
DictionaryNameFinder can only be deployed on un-annotated text...To be
honest i don't see any other reason why it won't use the dictionary
instead of the model since both classes conform to the same interface -
otherwise Java would complain!
Any ideas?
Jim
p.s: i 'm not sure if i can call this a bug or a massive
improvement...it all started when i started thinking how i can evaluate
my trained model when it joins forces with the dictionary...i can see
with my eyes there is some improvement but it is crucial that i get some
numbers...
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
I am not sure that if we set it to 1.5.2 we are supposed to generate a
fix for that version. I think it is not the case. We should have
strong reasons for that, and you are using trunk anyway.
On Sat, Mar 17, 2012 at 3:30 PM, Jim - FooBar(); <ji...@gmail.com> wrote:
> I opened JIRA issue but i may have done something wrong...
> I 'm stating that it affects version 1.5.2 while in fact i'm using the
> latest code from svn (checkout a version this morning)...if by any chance it
> used to work with 1.5.2 then my JIRA ticket is inaccurate...
>
> Jim
>
>
> On 17/03/12 18:22, Jim - FooBar(); wrote:
>>
>> Hey William,
>>
>> That sounds easy so, good news...I'll open a JIRA for it now.
>>
>> Jim
>>
>> On 17/03/12 18:16, william.colen@gmail.com wrote:
>>>
>>> I found the problem. The NameFinderEvaluator expects typed spans, but
>>> the DictionaryNameFinder outputs the old untyped spans. For now we
>>> should fix it by setting the span types to default.
>>>
>>> Can you please open a Jira ?
>>>
>>> Thank you,
>>> William
>>>
>>>
>>> On Sat, Mar 17, 2012 at 11:01 AM, william.colen@gmail.com
>>> <wi...@gmail.com> wrote:
>>>>
>>>> Thank you, I read it too fast.
>>>>
>>>> I will try to check what is wrong latter today.
>>>>
>>>> On Sat, Mar 17, 2012 at 10:55 AM, Jim - FooBar();<ji...@gmail.com>
>>>> wrote:
>>>>>
>>>>> On 17/03/12 13:37, Jim - FooBar(); wrote:
>>>>>>
>>>>>> I'd like to start working on integrating the 2 but i can't, unless
>>>>>> I can be sure that both evaluations work, at least independently.
>>>>>
>>>>>
>>>>> Hey William,
>>>>>
>>>>> maybe the above phrase confused you...
>>>>> by "both evaluations" i meant for the DictionaryNameFinder& the
>>>>> NameFinderME objects. So far i've only managed to do both regular
>>>>> evaluation and cross-valdation for the model but i can't do any kind of
>>>>> evaluation on the Dictionary.
>>>>>
>>>>> (of course only one type of evaluation makes sense for the dictionary
>>>>> and
>>>>> that is the regular one, but that always returns 0s and -1s).
>>>>>
>>>>> Hope that helps
>>>>>
>>>>> Jim
>>
>>
>
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "Jim - FooBar();" <ji...@gmail.com>.
I opened JIRA issue but i may have done something wrong...
I 'm stating that it affects version 1.5.2 while in fact i'm using the
latest code from svn (checkout a version this morning)...if by any
chance it used to work with 1.5.2 then my JIRA ticket is inaccurate...
Jim
On 17/03/12 18:22, Jim - FooBar(); wrote:
> Hey William,
>
> That sounds easy so, good news...I'll open a JIRA for it now.
>
> Jim
>
> On 17/03/12 18:16, william.colen@gmail.com wrote:
>> I found the problem. The NameFinderEvaluator expects typed spans, but
>> the DictionaryNameFinder outputs the old untyped spans. For now we
>> should fix it by setting the span types to default.
>>
>> Can you please open a Jira ?
>>
>> Thank you,
>> William
>>
>>
>> On Sat, Mar 17, 2012 at 11:01 AM, william.colen@gmail.com
>> <wi...@gmail.com> wrote:
>>> Thank you, I read it too fast.
>>>
>>> I will try to check what is wrong latter today.
>>>
>>> On Sat, Mar 17, 2012 at 10:55 AM, Jim -
>>> FooBar();<ji...@gmail.com> wrote:
>>>> On 17/03/12 13:37, Jim - FooBar(); wrote:
>>>>> I'd like to start working on integrating the 2 but i can't, unless
>>>>> I can be sure that both evaluations work, at least independently.
>>>>
>>>> Hey William,
>>>>
>>>> maybe the above phrase confused you...
>>>> by "both evaluations" i meant for the DictionaryNameFinder& the
>>>> NameFinderME objects. So far i've only managed to do both regular
>>>> evaluation and cross-valdation for the model but i can't do any
>>>> kind of
>>>> evaluation on the Dictionary.
>>>>
>>>> (of course only one type of evaluation makes sense for the
>>>> dictionary and
>>>> that is the regular one, but that always returns 0s and -1s).
>>>>
>>>> Hope that helps
>>>>
>>>> Jim
>
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "Jim - FooBar();" <ji...@gmail.com>.
Hey William,
That sounds easy so, good news...I'll open a JIRA for it now.
Jim
On 17/03/12 18:16, william.colen@gmail.com wrote:
> I found the problem. The NameFinderEvaluator expects typed spans, but
> the DictionaryNameFinder outputs the old untyped spans. For now we
> should fix it by setting the span types to default.
>
> Can you please open a Jira ?
>
> Thank you,
> William
>
>
> On Sat, Mar 17, 2012 at 11:01 AM, william.colen@gmail.com
> <wi...@gmail.com> wrote:
>> Thank you, I read it too fast.
>>
>> I will try to check what is wrong latter today.
>>
>> On Sat, Mar 17, 2012 at 10:55 AM, Jim - FooBar();<ji...@gmail.com> wrote:
>>> On 17/03/12 13:37, Jim - FooBar(); wrote:
>>>> I'd like to start working on integrating the 2 but i can't, unless
>>>> I can be sure that both evaluations work, at least independently.
>>>
>>> Hey William,
>>>
>>> maybe the above phrase confused you...
>>> by "both evaluations" i meant for the DictionaryNameFinder& the
>>> NameFinderME objects. So far i've only managed to do both regular
>>> evaluation and cross-valdation for the model but i can't do any kind of
>>> evaluation on the Dictionary.
>>>
>>> (of course only one type of evaluation makes sense for the dictionary and
>>> that is the regular one, but that always returns 0s and -1s).
>>>
>>> Hope that helps
>>>
>>> Jim
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
I found the problem. The NameFinderEvaluator expects typed spans, but
the DictionaryNameFinder outputs the old untyped spans. For now we
should fix it by setting the span types to default.
Can you please open a Jira ?
Thank you,
William
On Sat, Mar 17, 2012 at 11:01 AM, william.colen@gmail.com
<wi...@gmail.com> wrote:
> Thank you, I read it too fast.
>
> I will try to check what is wrong latter today.
>
> On Sat, Mar 17, 2012 at 10:55 AM, Jim - FooBar(); <ji...@gmail.com> wrote:
>> On 17/03/12 13:37, Jim - FooBar(); wrote:
>>>
>>> I'd like to start working on integrating the 2 but i can't, unless
>>> I can be sure that both evaluations work, at least independently.
>>
>>
>> Hey William,
>>
>> maybe the above phrase confused you...
>> by "both evaluations" i meant for the DictionaryNameFinder & the
>> NameFinderME objects. So far i've only managed to do both regular
>> evaluation and cross-valdation for the model but i can't do any kind of
>> evaluation on the Dictionary.
>>
>> (of course only one type of evaluation makes sense for the dictionary and
>> that is the regular one, but that always returns 0s and -1s).
>>
>> Hope that helps
>>
>> Jim
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
Thank you, I read it too fast.
I will try to check what is wrong latter today.
On Sat, Mar 17, 2012 at 10:55 AM, Jim - FooBar(); <ji...@gmail.com> wrote:
> On 17/03/12 13:37, Jim - FooBar(); wrote:
>>
>> I'd like to start working on integrating the 2 but i can't, unless
>> I can be sure that both evaluations work, at least independently.
>
>
> Hey William,
>
> maybe the above phrase confused you...
> by "both evaluations" i meant for the DictionaryNameFinder & the
> NameFinderME objects. So far i've only managed to do both regular
> evaluation and cross-valdation for the model but i can't do any kind of
> evaluation on the Dictionary.
>
> (of course only one type of evaluation makes sense for the dictionary and
> that is the regular one, but that always returns 0s and -1s).
>
> Hope that helps
>
> Jim
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "Jim - FooBar();" <ji...@gmail.com>.
On 17/03/12 13:37, Jim - FooBar(); wrote:
> I'd like to start working on integrating the 2 but i can't, unless
> I can be sure that both evaluations work, at least independently.
Hey William,
maybe the above phrase confused you...
by "both evaluations" i meant for the DictionaryNameFinder & the
NameFinderME objects. So far i've only managed to do both regular
evaluation and cross-valdation for the model but i can't do any kind of
evaluation on the Dictionary.
(of course only one type of evaluation makes sense for the dictionary
and that is the regular one, but that always returns 0s and -1s).
Hope that helps
Jim
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "Jim - FooBar();" <ji...@gmail.com>.
I don't need cross-validation, i only need regular evaluation....that
was the confusion...
There is nothing to train with the DictionaryNameFInder, it doesn't make
any sense to use the cross-valdiator...
Jim
On 17/03/12 13:30, william.colen@gmail.com wrote:
> Why do you need cross-validation? Are you creating the dictionary
> dynamically from the corpus?
>
>
> On Sat, Mar 17, 2012 at 10:07 AM, Jim - FooBar();<ji...@gmail.com> wrote:
>> Hello again,
>>
>> basically, with the dictionary almost sorted, i'd like to return to a
>> previous issue that wasn't quite resolved. I made a mistake and posted that
>> i couldn't use the *cross-validator* with the dictionary, while in fact i
>> meant to say that i can't use the *regular evaluator* with the
>> dictionaryNameFinder...it always returns precission=0, recall=0, f-score=-1,
>> regardless of finding loads of entities... Has anyone tried doing that?
>>
>> I'm not sure why this happens - from what i can see DictionaryNameFinder
>> implements TokenNameFinder (as does NameFinderME) and that is all the
>> TokenNameFinderEvaluator needs!I would expect this to work but it
>> doesn't...I'd like to start working on integrating the 2 but i can't, unless
>> I can be sure that both evaluations work, at least independently.
>>
>> Jim
>>
>>
>> On 13/03/12 17:00, Jim - FooBar(); wrote:
>>> Ooops i'm really sorry...I am using the regular evaluator not the
>>> cross-validator...The previous message should have been about the standard
>>> evaluator...My bad! There is simply no point in using the cross-validator
>>> with the dictionary - there is nothing to train! I apologise for the mistake
>>> it wont happen again! The thing is i'm pretty stressed out!
>>>
>>> Jim
>>>
>>>
>>>
>>> On 13/03/12 16:51, Jörn Kottmann wrote:
>>>> You only do cross validation because you need to take some data out to
>>>> train your model on. That is why you can pass in all the training
>>>> parameters
>>>> to the TokenNameFinderCrossValidator.
>>>>
>>>> Maybe I am mistaken but it cannot take a TokenNameFinder object as an
>>>> argument, right?
>>>> Did you sub-classed it?
>>>>
>>>> Anyway since the DictionaryNameFinder cannot be trained you should
>>>> just use the evaluators they are simpler and give you the same result.
>>>>
>>>> Jörn
>>>>
>>>> On 03/13/2012 05:34 PM, Jim - FooBar(); wrote:
>>>>> Hey guys,
>>>>>
>>>>> First of all i can imagine you must be sick and tired of me reporting a
>>>>> bug or improvement every single day!That is the nature of open-source though
>>>>> isn't it? :-)
>>>>>
>>>>> Today's issue came literally out of nowhere! Again it has to do with
>>>>> cross-validation but with the DictionaryNameFinder this time - NOT the
>>>>> maxent model...Ok, here goes:
>>>>>
>>>>> Basically, both the maxentNameFinder and the DictionaryNameFinder can do
>>>>> NER. Also both classes implement TokenNameFinder so from Java's perspective
>>>>> either can be passed as argument to the TokenNameFinderCrossValidator
>>>>> constructor...However, i tried doing that this morning, in an effort to get
>>>>> some numbers my dictionary, and all i get is 0 precision, 0 recall and -1
>>>>> FMeasure, regardless of finding loads of drugs! I think (not 100% sure) the
>>>>> problem is that the CrossValidator expects annotated text (in order to
>>>>> verify) but the DictionaryNameFinder can only be deployed on un-annotated
>>>>> text...To be honest i don't see any other reason why it won't use the
>>>>> dictionary instead of the model since both classes conform to the same
>>>>> interface - otherwise Java would complain!
>>>>>
>>>>> Any ideas?
>>>>>
>>>>>
>>>>> Jim
>>>>>
>>>>> p.s: i 'm not sure if i can call this a bug or a massive
>>>>> improvement...it all started when i started thinking how i can evaluate my
>>>>> trained model when it joins forces with the dictionary...i can see with my
>>>>> eyes there is some improvement but it is crucial that i get some numbers...
>>>>
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
Why do you need cross-validation? Are you creating the dictionary
dynamically from the corpus?
On Sat, Mar 17, 2012 at 10:07 AM, Jim - FooBar(); <ji...@gmail.com> wrote:
> Hello again,
>
> basically, with the dictionary almost sorted, i'd like to return to a
> previous issue that wasn't quite resolved. I made a mistake and posted that
> i couldn't use the *cross-validator* with the dictionary, while in fact i
> meant to say that i can't use the *regular evaluator* with the
> dictionaryNameFinder...it always returns precission=0, recall=0, f-score=-1,
> regardless of finding loads of entities... Has anyone tried doing that?
>
> I'm not sure why this happens - from what i can see DictionaryNameFinder
> implements TokenNameFinder (as does NameFinderME) and that is all the
> TokenNameFinderEvaluator needs!I would expect this to work but it
> doesn't...I'd like to start working on integrating the 2 but i can't, unless
> I can be sure that both evaluations work, at least independently.
>
> Jim
>
>
> On 13/03/12 17:00, Jim - FooBar(); wrote:
>>
>> Ooops i'm really sorry...I am using the regular evaluator not the
>> cross-validator...The previous message should have been about the standard
>> evaluator...My bad! There is simply no point in using the cross-validator
>> with the dictionary - there is nothing to train! I apologise for the mistake
>> it wont happen again! The thing is i'm pretty stressed out!
>>
>> Jim
>>
>>
>>
>> On 13/03/12 16:51, Jörn Kottmann wrote:
>>>
>>> You only do cross validation because you need to take some data out to
>>> train your model on. That is why you can pass in all the training
>>> parameters
>>> to the TokenNameFinderCrossValidator.
>>>
>>> Maybe I am mistaken but it cannot take a TokenNameFinder object as an
>>> argument, right?
>>> Did you sub-classed it?
>>>
>>> Anyway since the DictionaryNameFinder cannot be trained you should
>>> just use the evaluators they are simpler and give you the same result.
>>>
>>> Jörn
>>>
>>> On 03/13/2012 05:34 PM, Jim - FooBar(); wrote:
>>>>
>>>> Hey guys,
>>>>
>>>> First of all i can imagine you must be sick and tired of me reporting a
>>>> bug or improvement every single day!That is the nature of open-source though
>>>> isn't it? :-)
>>>>
>>>> Today's issue came literally out of nowhere! Again it has to do with
>>>> cross-validation but with the DictionaryNameFinder this time - NOT the
>>>> maxent model...Ok, here goes:
>>>>
>>>> Basically, both the maxentNameFinder and the DictionaryNameFinder can do
>>>> NER. Also both classes implement TokenNameFinder so from Java's perspective
>>>> either can be passed as argument to the TokenNameFinderCrossValidator
>>>> constructor...However, i tried doing that this morning, in an effort to get
>>>> some numbers my dictionary, and all i get is 0 precision, 0 recall and -1
>>>> FMeasure, regardless of finding loads of drugs! I think (not 100% sure) the
>>>> problem is that the CrossValidator expects annotated text (in order to
>>>> verify) but the DictionaryNameFinder can only be deployed on un-annotated
>>>> text...To be honest i don't see any other reason why it won't use the
>>>> dictionary instead of the model since both classes conform to the same
>>>> interface - otherwise Java would complain!
>>>>
>>>> Any ideas?
>>>>
>>>>
>>>> Jim
>>>>
>>>> p.s: i 'm not sure if i can call this a bug or a massive
>>>> improvement...it all started when i started thinking how i can evaluate my
>>>> trained model when it joins forces with the dictionary...i can see with my
>>>> eyes there is some improvement but it is crucial that i get some numbers...
>>>
>>>
>>
>
Re: DictionaryNameFinder evaluation always returns 0, 0, -1
Posted by "Jim - FooBar();" <ji...@gmail.com>.
Hello again,
basically, with the dictionary almost sorted, i'd like to return to a
previous issue that wasn't quite resolved. I made a mistake and posted
that i couldn't use the *cross-validator* with the dictionary, while in
fact i meant to say that i can't use the *regular evaluator* with the
dictionaryNameFinder...it always returns precission=0, recall=0,
f-score=-1, regardless of finding loads of entities... Has anyone tried
doing that?
I'm not sure why this happens - from what i can see DictionaryNameFinder
implements TokenNameFinder (as does NameFinderME) and that is all the
TokenNameFinderEvaluator needs!I would expect this to work but it
doesn't...I'd like to start working on integrating the 2 but i can't,
unless I can be sure that both evaluations work, at least independently.
Jim
On 13/03/12 17:00, Jim - FooBar(); wrote:
> Ooops i'm really sorry...I am using the regular evaluator not the
> cross-validator...The previous message should have been about the
> standard evaluator...My bad! There is simply no point in using the
> cross-validator with the dictionary - there is nothing to train! I
> apologise for the mistake it wont happen again! The thing is i'm
> pretty stressed out!
>
> Jim
>
>
>
> On 13/03/12 16:51, Jörn Kottmann wrote:
>> You only do cross validation because you need to take some data out to
>> train your model on. That is why you can pass in all the training
>> parameters
>> to the TokenNameFinderCrossValidator.
>>
>> Maybe I am mistaken but it cannot take a TokenNameFinder object as an
>> argument, right?
>> Did you sub-classed it?
>>
>> Anyway since the DictionaryNameFinder cannot be trained you should
>> just use the evaluators they are simpler and give you the same result.
>>
>> Jörn
>>
>> On 03/13/2012 05:34 PM, Jim - FooBar(); wrote:
>>> Hey guys,
>>>
>>> First of all i can imagine you must be sick and tired of me
>>> reporting a bug or improvement every single day!That is the nature
>>> of open-source though isn't it? :-)
>>>
>>> Today's issue came literally out of nowhere! Again it has to do with
>>> cross-validation but with the DictionaryNameFinder this time - NOT
>>> the maxent model...Ok, here goes:
>>>
>>> Basically, both the maxentNameFinder and the DictionaryNameFinder
>>> can do NER. Also both classes implement TokenNameFinder so from
>>> Java's perspective either can be passed as argument to the
>>> TokenNameFinderCrossValidator constructor...However, i tried doing
>>> that this morning, in an effort to get some numbers my dictionary,
>>> and all i get is 0 precision, 0 recall and -1 FMeasure, regardless
>>> of finding loads of drugs! I think (not 100% sure) the problem is
>>> that the CrossValidator expects annotated text (in order to verify)
>>> but the DictionaryNameFinder can only be deployed on un-annotated
>>> text...To be honest i don't see any other reason why it won't use
>>> the dictionary instead of the model since both classes conform to
>>> the same interface - otherwise Java would complain!
>>>
>>> Any ideas?
>>>
>>>
>>> Jim
>>>
>>> p.s: i 'm not sure if i can call this a bug or a massive
>>> improvement...it all started when i started thinking how i can
>>> evaluate my trained model when it joins forces with the
>>> dictionary...i can see with my eyes there is some improvement but it
>>> is crucial that i get some numbers...
>>
>
Re: DictionaryNameFinder cross-validation?
Posted by "Jim - FooBar();" <ji...@gmail.com>.
Ooops i'm really sorry...I am using the regular evaluator not the
cross-validator...The previous message should have been about the
standard evaluator...My bad! There is simply no point in using the
cross-validator with the dictionary - there is nothing to train! I
apologise for the mistake it wont happen again! The thing is i'm pretty
stressed out!
Jim
On 13/03/12 16:51, Jörn Kottmann wrote:
> You only do cross validation because you need to take some data out to
> train your model on. That is why you can pass in all the training
> parameters
> to the TokenNameFinderCrossValidator.
>
> Maybe I am mistaken but it cannot take a TokenNameFinder object as an
> argument, right?
> Did you sub-classed it?
>
> Anyway since the DictionaryNameFinder cannot be trained you should
> just use the evaluators they are simpler and give you the same result.
>
> Jörn
>
> On 03/13/2012 05:34 PM, Jim - FooBar(); wrote:
>> Hey guys,
>>
>> First of all i can imagine you must be sick and tired of me reporting
>> a bug or improvement every single day!That is the nature of
>> open-source though isn't it? :-)
>>
>> Today's issue came literally out of nowhere! Again it has to do with
>> cross-validation but with the DictionaryNameFinder this time - NOT
>> the maxent model...Ok, here goes:
>>
>> Basically, both the maxentNameFinder and the DictionaryNameFinder can
>> do NER. Also both classes implement TokenNameFinder so from Java's
>> perspective either can be passed as argument to the
>> TokenNameFinderCrossValidator constructor...However, i tried doing
>> that this morning, in an effort to get some numbers my dictionary,
>> and all i get is 0 precision, 0 recall and -1 FMeasure, regardless of
>> finding loads of drugs! I think (not 100% sure) the problem is that
>> the CrossValidator expects annotated text (in order to verify) but
>> the DictionaryNameFinder can only be deployed on un-annotated
>> text...To be honest i don't see any other reason why it won't use the
>> dictionary instead of the model since both classes conform to the
>> same interface - otherwise Java would complain!
>>
>> Any ideas?
>>
>>
>> Jim
>>
>> p.s: i 'm not sure if i can call this a bug or a massive
>> improvement...it all started when i started thinking how i can
>> evaluate my trained model when it joins forces with the
>> dictionary...i can see with my eyes there is some improvement but it
>> is crucial that i get some numbers...
>
Re: DictionaryNameFinder cross-validation?
Posted by Jörn Kottmann <ko...@gmail.com>.
You only do cross validation because you need to take some data out to
train your model on. That is why you can pass in all the training parameters
to the TokenNameFinderCrossValidator.
Maybe I am mistaken but it cannot take a TokenNameFinder object as an
argument, right?
Did you sub-classed it?
Anyway since the DictionaryNameFinder cannot be trained you should
just use the evaluators they are simpler and give you the same result.
Jörn
On 03/13/2012 05:34 PM, Jim - FooBar(); wrote:
> Hey guys,
>
> First of all i can imagine you must be sick and tired of me reporting
> a bug or improvement every single day!That is the nature of
> open-source though isn't it? :-)
>
> Today's issue came literally out of nowhere! Again it has to do with
> cross-validation but with the DictionaryNameFinder this time - NOT the
> maxent model...Ok, here goes:
>
> Basically, both the maxentNameFinder and the DictionaryNameFinder can
> do NER. Also both classes implement TokenNameFinder so from Java's
> perspective either can be passed as argument to the
> TokenNameFinderCrossValidator constructor...However, i tried doing
> that this morning, in an effort to get some numbers my dictionary, and
> all i get is 0 precision, 0 recall and -1 FMeasure, regardless of
> finding loads of drugs! I think (not 100% sure) the problem is that
> the CrossValidator expects annotated text (in order to verify) but the
> DictionaryNameFinder can only be deployed on un-annotated text...To be
> honest i don't see any other reason why it won't use the dictionary
> instead of the model since both classes conform to the same interface
> - otherwise Java would complain!
>
> Any ideas?
>
>
> Jim
>
> p.s: i 'm not sure if i can call this a bug or a massive
> improvement...it all started when i started thinking how i can
> evaluate my trained model when it joins forces with the dictionary...i
> can see with my eyes there is some improvement but it is crucial that
> i get some numbers...