You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@opennlp.apache.org by Dimitrios <ji...@gmail.com> on 2012/03/08 17:10:17 UTC
strange NullPointerException when using the cross-validator...
Hi everyone,
Can someone please explain why i 'm getting this NPE when trying to use
the TokenNameFinderCrossValidator?
TokenNameFinderEvaluator seems to be working just fine but because i've
got very little training-data i'd like to do cross-validation as well...
*The code is as follows:*
public String cross-validate(String fname, int kFolds){
FileInputStream sampleDataIn = new FileInputStream(fname);
ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
new PlainTextByLineStream(
sampleDataIn.getChannel(), "UTF-8"));
TokenNameFinderCrossValidator judge = new
TokenNameFinderCrossValidator("en",
null,
new TrainingParameters(),
null,
null,
null);
judge.evaluate(sampleStream, kFolds);
FMeasure result = evaluator.getFMeasure();
return result.toString();
}
*The exception is thrown right after the first 100 iterations for the
first fold*:
NullPointerException
opennlp.tools.namefind.TokenNameFinderModel.<init>
(TokenNameFinderModel.java:90)
*If i omit the "new NameSampleDataStream()" (as shown in the docs) i get
a different error saying:*
ClassCastException java.lang.String cannot be cast to
opennlp.tools.namefind.NameSample
opennlp.tools.namefind.TokenNameFinderCrossValidator$NameToDocumentSampleStream.read
(TokenNameFinderCrossValidator.java:67)
What am i missing?
Thanks in advance...
Jim
Re: strange NullPointerException when using the cross-validator...
Posted by "Jim - FooBar();" <ji...@gmail.com>.
On 09/03/12 17:34, william.colen@gmail.com wrote:
> Although the JavaDoc states that you can pass in a null resource, looks
> like it causes the null pointer exception later.
>
> Please try passing an empty map instead of null.
Thanks a lot i'll try it now...
> Open a Jira issue so we can improve it.
I apologise but i'm not exactly sure how to do that...i had a look at:
https://issues.apache.org/jira/browse/OPENNLP
and i don't see any links for creating an issue...presumably one has to
log in and i can't...
apparently my username does not exist! tried my email but that doesn't
work either...
Jim
Re: Cross-Validator listeners problem...
Posted by Jörn Kottmann <ko...@gmail.com>.
Updated instructions are here:
http://opennlp.apache.org/mail-lists.html
Jörn
On 03/12/2012 07:05 PM, Jim - FooBar(); wrote:
> Have a look here:
> http://incubator.apache.org/opennlp/mail-lists.html
>
> Mind you, that openNLP is no longer "incubating" so be careful where
> you send the unsubscribe e-mail...you may have to omit the "incubator"
> bit...
>
> Hope that helps
>
> Jim
>
>
> On 12/03/12 17:00, Marcelo Lombardi wrote:
>> Hello,
>>
>> Could you please tell me how to get out of this list? Thank you!
>>
>> -----Mensaje original-----
>> De: Jim - FooBar(); [mailto:jimpil1985@gmail.com]
>> Enviado el: lunes, 12 de marzo de 2012 14:58
>> Para: users@opennlp.apache.org
>> Asunto: Re: Cross-Validator listeners problem...
>>
>> On 12/03/12 16:21, william.colen@gmail.com wrote:
>>> Probably another bug. Can you create a Jira? Please copy the stack
>>> trace.
>> Hmmm, there is no stack trace!!!
>> as i said the program just hangs - it doesn't crash - there are no
>> exceptions!
>>
>> I will create a Jira for it but i don't have any stack traces i'm
>> afraid...
>>
>> Jim
>>
>> p.s: the odd thing is this:
>>
>> the NameEvaluationErrorListener does work as expected during
>> cross-validation...In other words i can indeed see all the
>> misclassifications while the cross-validator is running! The problem
>> manifests as soon as the validator finishes with all k partitions and
>> its
>> time to calculate the f-score...Then it just hangs, but it only hangs
>> after
>> showing the errors for the last document of training - not
>> before...this is
>> very strange!!!
>>
>
>
Re: Cross-Validator listeners problem...
Posted by "Jim - FooBar();" <ji...@gmail.com>.
Have a look here:
http://incubator.apache.org/opennlp/mail-lists.html
Mind you, that openNLP is no longer "incubating" so be careful where you
send the unsubscribe e-mail...you may have to omit the "incubator" bit...
Hope that helps
Jim
On 12/03/12 17:00, Marcelo Lombardi wrote:
> Hello,
>
> Could you please tell me how to get out of this list? Thank you!
>
> -----Mensaje original-----
> De: Jim - FooBar(); [mailto:jimpil1985@gmail.com]
> Enviado el: lunes, 12 de marzo de 2012 14:58
> Para: users@opennlp.apache.org
> Asunto: Re: Cross-Validator listeners problem...
>
> On 12/03/12 16:21, william.colen@gmail.com wrote:
>> Probably another bug. Can you create a Jira? Please copy the stack trace.
> Hmmm, there is no stack trace!!!
> as i said the program just hangs - it doesn't crash - there are no
> exceptions!
>
> I will create a Jira for it but i don't have any stack traces i'm afraid...
>
> Jim
>
> p.s: the odd thing is this:
>
> the NameEvaluationErrorListener does work as expected during
> cross-validation...In other words i can indeed see all the
> misclassifications while the cross-validator is running! The problem
> manifests as soon as the validator finishes with all k partitions and its
> time to calculate the f-score...Then it just hangs, but it only hangs after
> showing the errors for the last document of training - not before...this is
> very strange!!!
>
RE: Cross-Validator listeners problem...
Posted by Marcelo Lombardi <ma...@gmail.com>.
Hello,
Could you please tell me how to get out of this list? Thank you!
-----Mensaje original-----
De: Jim - FooBar(); [mailto:jimpil1985@gmail.com]
Enviado el: lunes, 12 de marzo de 2012 14:58
Para: users@opennlp.apache.org
Asunto: Re: Cross-Validator listeners problem...
On 12/03/12 16:21, william.colen@gmail.com wrote:
> Probably another bug. Can you create a Jira? Please copy the stack trace.
Hmmm, there is no stack trace!!!
as i said the program just hangs - it doesn't crash - there are no
exceptions!
I will create a Jira for it but i don't have any stack traces i'm afraid...
Jim
p.s: the odd thing is this:
the NameEvaluationErrorListener does work as expected during
cross-validation...In other words i can indeed see all the
misclassifications while the cross-validator is running! The problem
manifests as soon as the validator finishes with all k partitions and its
time to calculate the f-score...Then it just hangs, but it only hangs after
showing the errors for the last document of training - not before...this is
very strange!!!
Re: Cross-Validator listeners problem...
Posted by "Jim - FooBar();" <ji...@gmail.com>.
On 12/03/12 16:21, william.colen@gmail.com wrote:
> Probably another bug. Can you create a Jira? Please copy the stack trace.
Hmmm, there is no stack trace!!!
as i said the program just hangs - it doesn't crash - there are no
exceptions!
I will create a Jira for it but i don't have any stack traces i'm afraid...
Jim
p.s: the odd thing is this:
the NameEvaluationErrorListener does work as expected during
cross-validation...In other words i can indeed see all the
misclassifications while the cross-validator is running! The problem
manifests as soon as the validator finishes with all k partitions and
its time to calculate the f-score...Then it just hangs, but it only
hangs after showing the errors for the last document of training - not
before...this is very strange!!!
Re: Cross-Validator listeners problem...
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
Hi Jim,
Probably another bug. Can you create a Jira? Please copy the stack trace.
Thank you,
William
On Mon, Mar 12, 2012 at 11:57 AM, Dimitrios <ji...@gmail.com> wrote:
> Hello again,
>
> Ok so the cross-validator needs an empty Map in order not to throw a NPE.
> The correct method is:
>
> public String cross-validate(String fname, int kFolds){
>>
>> FileInputStream sampleDataIn = new FileInputStream(fname);
>> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>>
>> new PlainTextByLineStream(
>>
>> sampleDataIn.getChannel(), "UTF-8"));
>> TokenNameFinderCrossValidator judge =
>> new TokenNameFinderCrossValidator(**"en", null, new
>> TrainingParameters(), null, new HashMap(), null);
>>
>> judge.evaluate(sampleStream, kFolds);
>> FMeasure result = evaluator.getFMeasure();
>> return result.toString();
>>
>> }
>>
>
>
> However if i try to pass a "NameEvaluationErrorListener" as the last
> parameter instead of null the cross-validator just hangs before i can do
> getFMeasure()!!! Actually it hangs right after printing the
> misclassifications for the last document in the training data...If i leave
> the listeners parameter as null or with a different listener (e.g
> TokenNameFinderDetailedFMeasur**eListener) it works just fine meaning i
> can get my FMeasure and print it...
>
> Any ideas as to what could be the problem with the
> NameEvaluationErrorListener which is indeed the most useful listener for
> cross-validation?
>
> Thanks in advance...
>
> Jim
>
>
> On 09/03/12 17:34, william.colen@gmail.com wrote:
>
>> Hi,
>>
>> Although the JavaDoc states that you can pass in a null resource, looks
>> like it causes the null pointer exception later.
>>
>> Please try passing an empty map instead of null.
>>
>> Open a Jira issue so we can improve it.
>>
>> Thank you,
>> William
>>
>> On Thu, Mar 8, 2012 at 6:28 PM, Jim - FooBar();<jimpil1985@gmail.com**
>> >wrote:
>>
>> I should clarify that i 'm using the latest code from svn...
>>>
>>> Any ideas?
>>>
>>>
>>>
>>> On 08/03/12 16:10, Dimitrios wrote:
>>>
>>> Hi everyone,
>>>>
>>>> Can someone please explain why i 'm getting this NPE when trying to use
>>>> the TokenNameFinderCrossValidator?
>>>> TokenNameFinderEvaluator seems to be working just fine but because i've
>>>> got very little training-data i'd like to do cross-validation as well...
>>>>
>>>> *The code is as follows:*
>>>>
>>>> public String cross-validate(String fname, int kFolds){
>>>>
>>>> FileInputStream sampleDataIn = new FileInputStream(fname);
>>>> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>>>>
>>>> new PlainTextByLineStream(
>>>>
>>>> sampleDataIn.getChannel(), "UTF-8"));
>>>> TokenNameFinderCrossValidator judge = new TokenNameFinderCrossValidator(
>>>> ***
>>>> *"en",
>>>>
>>>> null,
>>>>
>>>> new TrainingParameters(),
>>>>
>>>> null,
>>>>
>>>> null,
>>>>
>>>> null);
>>>> judge.evaluate(sampleStream, kFolds);
>>>> FMeasure result = evaluator.getFMeasure();
>>>> return result.toString();
>>>>
>>>> }
>>>>
>>>> *The exception is thrown right after the first 100 iterations for the
>>>> first fold*:
>>>>
>>>> NullPointerException opennlp.tools.namefind.****
>>>> TokenNameFinderModel.<init>
>>>> (TokenNameFinderModel.java:90)
>>>>
>>>> *If i omit the "new NameSampleDataStream()" (as shown in the docs) i get
>>>> a different error saying:*
>>>>
>>>> ClassCastException java.lang.String cannot be cast to
>>>> opennlp.tools.namefind.****NameSample opennlp.tools.namefind.**
>>>> TokenNameFinderCrossValidator$****NameToDocumentSampleStream.****read
>>>> (**
>>>> TokenNameFinderCrossValidator.****java:67)
>>>>
>>>> What am i missing?
>>>>
>>>> Thanks in advance...
>>>>
>>>> Jim
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>
Cross-Validator listeners problem...
Posted by Dimitrios <ji...@gmail.com>.
Hello again,
Ok so the cross-validator needs an empty Map in order not to throw a
NPE. The correct method is:
> public String cross-validate(String fname, int kFolds){
>
> FileInputStream sampleDataIn = new FileInputStream(fname);
> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>
> new PlainTextByLineStream(
>
> sampleDataIn.getChannel(), "UTF-8"));
> TokenNameFinderCrossValidator judge =
> new TokenNameFinderCrossValidator("en", null, new
> TrainingParameters(), null, new HashMap(), null);
>
> judge.evaluate(sampleStream, kFolds);
> FMeasure result = evaluator.getFMeasure();
> return result.toString();
>
> }
However if i try to pass a "NameEvaluationErrorListener" as the last
parameter instead of null the cross-validator just hangs before i can
do getFMeasure()!!! Actually it hangs right after printing the
misclassifications for the last document in the training data...If i
leave the listeners parameter as null or with a different listener (e.g
TokenNameFinderDetailedFMeasureListener) it works just fine meaning i
can get my FMeasure and print it...
Any ideas as to what could be the problem with the
NameEvaluationErrorListener which is indeed the most useful listener for
cross-validation?
Thanks in advance...
Jim
On 09/03/12 17:34, william.colen@gmail.com wrote:
> Hi,
>
> Although the JavaDoc states that you can pass in a null resource, looks
> like it causes the null pointer exception later.
>
> Please try passing an empty map instead of null.
>
> Open a Jira issue so we can improve it.
>
> Thank you,
> William
>
> On Thu, Mar 8, 2012 at 6:28 PM, Jim - FooBar();<ji...@gmail.com>wrote:
>
>> I should clarify that i 'm using the latest code from svn...
>>
>> Any ideas?
>>
>>
>>
>> On 08/03/12 16:10, Dimitrios wrote:
>>
>>> Hi everyone,
>>>
>>> Can someone please explain why i 'm getting this NPE when trying to use
>>> the TokenNameFinderCrossValidator?
>>> TokenNameFinderEvaluator seems to be working just fine but because i've
>>> got very little training-data i'd like to do cross-validation as well...
>>>
>>> *The code is as follows:*
>>>
>>> public String cross-validate(String fname, int kFolds){
>>>
>>> FileInputStream sampleDataIn = new FileInputStream(fname);
>>> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>>>
>>> new PlainTextByLineStream(
>>>
>>> sampleDataIn.getChannel(), "UTF-8"));
>>> TokenNameFinderCrossValidator judge = new TokenNameFinderCrossValidator(*
>>> *"en",
>>>
>>> null,
>>>
>>> new TrainingParameters(),
>>>
>>> null,
>>>
>>> null,
>>>
>>> null);
>>> judge.evaluate(sampleStream, kFolds);
>>> FMeasure result = evaluator.getFMeasure();
>>> return result.toString();
>>>
>>> }
>>>
>>> *The exception is thrown right after the first 100 iterations for the
>>> first fold*:
>>>
>>> NullPointerException opennlp.tools.namefind.**TokenNameFinderModel.<init>
>>> (TokenNameFinderModel.java:90)
>>>
>>> *If i omit the "new NameSampleDataStream()" (as shown in the docs) i get
>>> a different error saying:*
>>>
>>> ClassCastException java.lang.String cannot be cast to
>>> opennlp.tools.namefind.**NameSample opennlp.tools.namefind.**
>>> TokenNameFinderCrossValidator$**NameToDocumentSampleStream.**read (**
>>> TokenNameFinderCrossValidator.**java:67)
>>>
>>> What am i missing?
>>>
>>> Thanks in advance...
>>>
>>> Jim
>>>
>>>
>>>
>>>
>>>
Re: strange NullPointerException when using the cross-validator...
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
Thank you, Jim.
To create a new issue you will have to sign up for an account. Just follow
the to 'Log in' link on top right of the page.
After log in you will see a new 'create issue' link.
On Fri, Mar 9, 2012 at 2:57 PM, Jim - FooBar(); <ji...@gmail.com>wrote:
> On 09/03/12 17:34, william.colen@gmail.com wrote:
>
>> Please try passing an empty map instead of null.
>>
>
> Indeed it works now! it does not stop after first iteration any more...
> thanks a million!
> cannot wait for version 1.6...
>
> btw congratulations on the upgrade from the incubator!!!
>
> Jim
>
Re: strange NullPointerException when using the cross-validator...
Posted by "Jim - FooBar();" <ji...@gmail.com>.
On 09/03/12 17:34, william.colen@gmail.com wrote:
> Please try passing an empty map instead of null.
Indeed it works now! it does not stop after first iteration any more...
thanks a million!
cannot wait for version 1.6...
btw congratulations on the upgrade from the incubator!!!
Jim
Re: strange NullPointerException when using the cross-validator...
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
Hi,
Although the JavaDoc states that you can pass in a null resource, looks
like it causes the null pointer exception later.
Please try passing an empty map instead of null.
Open a Jira issue so we can improve it.
Thank you,
William
On Thu, Mar 8, 2012 at 6:28 PM, Jim - FooBar(); <ji...@gmail.com>wrote:
> I should clarify that i 'm using the latest code from svn...
>
> Any ideas?
>
>
>
> On 08/03/12 16:10, Dimitrios wrote:
>
>> Hi everyone,
>>
>> Can someone please explain why i 'm getting this NPE when trying to use
>> the TokenNameFinderCrossValidator?
>> TokenNameFinderEvaluator seems to be working just fine but because i've
>> got very little training-data i'd like to do cross-validation as well...
>>
>> *The code is as follows:*
>>
>> public String cross-validate(String fname, int kFolds){
>>
>> FileInputStream sampleDataIn = new FileInputStream(fname);
>> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>>
>> new PlainTextByLineStream(
>>
>> sampleDataIn.getChannel(), "UTF-8"));
>> TokenNameFinderCrossValidator judge = new TokenNameFinderCrossValidator(*
>> *"en",
>>
>> null,
>>
>> new TrainingParameters(),
>>
>> null,
>>
>> null,
>>
>> null);
>> judge.evaluate(sampleStream, kFolds);
>> FMeasure result = evaluator.getFMeasure();
>> return result.toString();
>>
>> }
>>
>> *The exception is thrown right after the first 100 iterations for the
>> first fold*:
>>
>> NullPointerException opennlp.tools.namefind.**TokenNameFinderModel.<init>
>> (TokenNameFinderModel.java:90)
>>
>> *If i omit the "new NameSampleDataStream()" (as shown in the docs) i get
>> a different error saying:*
>>
>> ClassCastException java.lang.String cannot be cast to
>> opennlp.tools.namefind.**NameSample opennlp.tools.namefind.**
>> TokenNameFinderCrossValidator$**NameToDocumentSampleStream.**read (**
>> TokenNameFinderCrossValidator.**java:67)
>>
>> What am i missing?
>>
>> Thanks in advance...
>>
>> Jim
>>
>>
>>
>>
>>
>
Re: strange NullPointerException when using the cross-validator...
Posted by "Jim - FooBar();" <ji...@gmail.com>.
I should clarify that i 'm using the latest code from svn...
Any ideas?
On 08/03/12 16:10, Dimitrios wrote:
> Hi everyone,
>
> Can someone please explain why i 'm getting this NPE when trying to
> use the TokenNameFinderCrossValidator?
> TokenNameFinderEvaluator seems to be working just fine but because
> i've got very little training-data i'd like to do cross-validation as
> well...
>
> *The code is as follows:*
>
> public String cross-validate(String fname, int kFolds){
>
> FileInputStream sampleDataIn = new FileInputStream(fname);
> ObjectStream<NameSample> sampleStream = new NameSampleDataStream(
>
> new PlainTextByLineStream(
>
> sampleDataIn.getChannel(), "UTF-8"));
> TokenNameFinderCrossValidator judge = new
> TokenNameFinderCrossValidator("en",
> null,
> new
> TrainingParameters(),
> null,
> null,
> null);
> judge.evaluate(sampleStream, kFolds);
> FMeasure result = evaluator.getFMeasure();
> return result.toString();
>
> }
>
> *The exception is thrown right after the first 100 iterations for the
> first fold*:
>
> NullPointerException
> opennlp.tools.namefind.TokenNameFinderModel.<init>
> (TokenNameFinderModel.java:90)
>
> *If i omit the "new NameSampleDataStream()" (as shown in the docs) i
> get a different error saying:*
>
> ClassCastException java.lang.String cannot be cast to
> opennlp.tools.namefind.NameSample
> opennlp.tools.namefind.TokenNameFinderCrossValidator$NameToDocumentSampleStream.read
> (TokenNameFinderCrossValidator.java:67)
>
> What am i missing?
>
> Thanks in advance...
>
> Jim
>
>
>
>