You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ctakes.apache.org by "Miller, Timothy" <Ti...@childrens.harvard.edu> on 2018/03/05 20:46:54 UTC
Re: Negation attribute annotator always negative [EXTERNAL]
Hi Nicholas,
Good to hear from you! This is quite a surprising result, it is much
more typical in my experience for the negation module to miss a
negation (because of long distances), so I'd like to try to replicate
this. Do you get that result for each sentence at a time? Or only if
all three sentences are like that in the note (as in, separated by
newlines)?
What version of cTAKES? And can you let me know the exact pipeline
components? (Which negation annotator, are there any custom modules?)
Thanks
Tim
On Mon, 2018-03-05 at 01:03 +0100, Nicolas Paris wrote:
> Hello
>
> I have integrated the ctakes PolarityCleartkAnalysisEngine in a
> pipeline
> such:
> - sentence detection
> - token detection
> - pos detection
> - DiseaseDisorderMention annotation
> - negation detection
>
> My example text is:
> I have a disease.
> I do not have a disease.
> I have some disease.
>
> "Disease" is an instance of DiseaseDisorderMention.
>
> However, all my example are tagged to negative:
>
> 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> DECODING/EVAL: NEGATED//-1 [570-577]
> (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> DECODING/EVAL: NEGATED//-1 [594-601]
> (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> DECODING/EVAL: NEGATED//-1 [1099-1106]
> (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
>
>
> Am I missing something in the pipeline ?
>
> Thanks,
>
>
Re: Negation attribute annotator always negative [EXTERNAL]
Posted by Nicolas Paris <ni...@gmail.com>.
Thanks.
I probably use two negation modules.
The first one is based on the documentation there https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+4.0+-+NE+Contexts
It only produces 0 when I am using it (a ContextAnnotator)
The second one works quite well and is based on this code
org.apache.ctakes.dependency.parser.ae.ClearNLPDependencyParserAE;
It produces 1 and -1.
About the DiseaseDisorderMention versus IdentifiedAnnotation, I just
switched them to test, at the same piece of code level. The former
works, the other not.
Do you recommend to use the [1] negex based on cleartk ML ?
Thanks.
Le 06 mars 2018 à 12:28, Miller, Timothy écrivait :
> From your previous email it sounds like you maybe use two negation modules? Some older modules predate the use of standard definitions and so assign values in {0, -1}, while for the machine learning-based module [1] it uses the constants in CONST.java which have values {1,-1}. That's one thing to consider.
> Another is that they may operate over different classes. The machine-learning operates over IdentifiedAnnotations so it should give them all values of 1,-1, which includes the DiseaseDisorderMention class but many others as well. Is it possible that new IdentifiedAnnotation types are being created after the negation modules run?
> Tim
>
> [1] http://svn.apache.org/viewvc/ctakes/trunk/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java?view=markup
>
> ________________________________________
> From: Nicolas Paris <ni...@gmail.com>
> Sent: Monday, March 5, 2018 5:56 PM
> To: user@ctakes.apache.org
> Subject: Re: Negation attribute annotator always negative [EXTERNAL]
>
> Hi Tim
>
> I spotted a bug in my code. I wasn't annotating all my token as
> basic_tokens. Right now, this works as expected.
>
> Still, when I use DiseaseDisorderMention, it works as expected. However,
> when I am annotating as regular IdentifiedAnnotation, the polarity is
> always 0. Maybe this means this is the correct behavior (the
> documentation says FocusAnnotationClass = org.apache.ctakes.typesystem.type.textsem.IdentifiedAnnotation )
>
>
> Le 05 mars 2018 à 23:23, Nicolas Paris écrivait :
> > Hi Tim,
> >
> > I am using ctakes 4.0.0.
> >
> > The input text is a mimic note, and I added those example sentence
> > within the text. They are separated by newlines, and/or other sentences.
> >
> > I have tested two negation modules: one is a custom ContextAnnotator
> > configured as described there: https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_cTAKES-2B4.0-2B-2D-2BNE-2BContexts&d=DwIDAw&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=ERGntOhKdzSedh23cqrhpWrsgv9alDebzh-F52qnIDo&s=jOgRWwgFQrW3ZFU3eYoHCa-D_JVbnXHx9qklFYMpvf8&e=
> > The other is by using the default one:
> > /org/apache/ctakes/assertion/models/polarity/sharpi2b2mipacqnegex/model.jar
> > (I guess this is also a negation module too ?)
> >
> > The pipeline is based on :
> > - sentence detection -> stanford parser that produces ctakes sentence annotations
> > - token detection -> idem but produces ctakes basic token annotations
> > - pos detection -> opennlp mayo released within ctakes
> > - chunk detection -> opennlp released within ctakes, not sure what it is
> >
> > Interestingly, when I am annotating the "disease" word with
> > "DiseaseDisorderMention", the polarity is always 1.
> > But when I am annotating it with "IdentifiedAnnotation" the polarity is
> > always 0, withou any modification of the source code but the annotation
> > kind.
> >
> > My guess is I am missing an AE or something.
> >
> > Le 05 mars 2018 à 21:46, Miller, Timothy écrivait :
> > > Hi Nicholas,
> > > Good to hear from you! This is quite a surprising result, it is much
> > > more typical in my experience for the negation module to miss a
> > > negation (because of long distances), so I'd like to try to replicate
> > > this. Do you get that result for each sentence at a time? Or only if
> > > all three sentences are like that in the note (as in, separated by
> > > newlines)?
> > > What version of cTAKES? And can you let me know the exact pipeline
> > > components? (Which negation annotator, are there any custom modules?)
> > > Thanks
> > > Tim
> > >
> > > On Mon, 2018-03-05 at 01:03 +0100, Nicolas Paris wrote:
> > > > Hello
> > > >
> > > > I have integrated the ctakes PolarityCleartkAnalysisEngine in a
> > > > pipeline
> > > > such:
> > > > - sentence detection
> > > > - token detection
> > > > - pos detection
> > > > - DiseaseDisorderMention annotation
> > > > - negation detection
> > > >
> > > > My example text is:
> > > > I have a disease.
> > > > I do not have a disease.
> > > > I have some disease.
> > > >
> > > > "Disease" is an instance of DiseaseDisorderMention.
> > > >
> > > > However, all my example are tagged to negative:
> > > >
> > > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > > DECODING/EVAL: NEGATED//-1 [570-577]
> > > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > > DECODING/EVAL: NEGATED//-1 [594-601]
> > > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > > DECODING/EVAL: NEGATED//-1 [1099-1106]
> > > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > >
> > > >
> > > > Am I missing something in the pipeline ?
> > > >
> > > > Thanks,
> > > >
> > > >
Re: Negation attribute annotator always negative [EXTERNAL]
Posted by "Miller, Timothy" <Ti...@childrens.harvard.edu>.
From your previous email it sounds like you maybe use two negation modules? Some older modules predate the use of standard definitions and so assign values in {0, -1}, while for the machine learning-based module [1] it uses the constants in CONST.java which have values {1,-1}. That's one thing to consider.
Another is that they may operate over different classes. The machine-learning operates over IdentifiedAnnotations so it should give them all values of 1,-1, which includes the DiseaseDisorderMention class but many others as well. Is it possible that new IdentifiedAnnotation types are being created after the negation modules run?
Tim
[1] http://svn.apache.org/viewvc/ctakes/trunk/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java?view=markup
________________________________________
From: Nicolas Paris <ni...@gmail.com>
Sent: Monday, March 5, 2018 5:56 PM
To: user@ctakes.apache.org
Subject: Re: Negation attribute annotator always negative [EXTERNAL]
Hi Tim
I spotted a bug in my code. I wasn't annotating all my token as
basic_tokens. Right now, this works as expected.
Still, when I use DiseaseDisorderMention, it works as expected. However,
when I am annotating as regular IdentifiedAnnotation, the polarity is
always 0. Maybe this means this is the correct behavior (the
documentation says FocusAnnotationClass = org.apache.ctakes.typesystem.type.textsem.IdentifiedAnnotation )
Le 05 mars 2018 à 23:23, Nicolas Paris écrivait :
> Hi Tim,
>
> I am using ctakes 4.0.0.
>
> The input text is a mimic note, and I added those example sentence
> within the text. They are separated by newlines, and/or other sentences.
>
> I have tested two negation modules: one is a custom ContextAnnotator
> configured as described there: https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_cTAKES-2B4.0-2B-2D-2BNE-2BContexts&d=DwIDAw&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=ERGntOhKdzSedh23cqrhpWrsgv9alDebzh-F52qnIDo&s=jOgRWwgFQrW3ZFU3eYoHCa-D_JVbnXHx9qklFYMpvf8&e=
> The other is by using the default one:
> /org/apache/ctakes/assertion/models/polarity/sharpi2b2mipacqnegex/model.jar
> (I guess this is also a negation module too ?)
>
> The pipeline is based on :
> - sentence detection -> stanford parser that produces ctakes sentence annotations
> - token detection -> idem but produces ctakes basic token annotations
> - pos detection -> opennlp mayo released within ctakes
> - chunk detection -> opennlp released within ctakes, not sure what it is
>
> Interestingly, when I am annotating the "disease" word with
> "DiseaseDisorderMention", the polarity is always 1.
> But when I am annotating it with "IdentifiedAnnotation" the polarity is
> always 0, withou any modification of the source code but the annotation
> kind.
>
> My guess is I am missing an AE or something.
>
> Le 05 mars 2018 à 21:46, Miller, Timothy écrivait :
> > Hi Nicholas,
> > Good to hear from you! This is quite a surprising result, it is much
> > more typical in my experience for the negation module to miss a
> > negation (because of long distances), so I'd like to try to replicate
> > this. Do you get that result for each sentence at a time? Or only if
> > all three sentences are like that in the note (as in, separated by
> > newlines)?
> > What version of cTAKES? And can you let me know the exact pipeline
> > components? (Which negation annotator, are there any custom modules?)
> > Thanks
> > Tim
> >
> > On Mon, 2018-03-05 at 01:03 +0100, Nicolas Paris wrote:
> > > Hello
> > >
> > > I have integrated the ctakes PolarityCleartkAnalysisEngine in a
> > > pipeline
> > > such:
> > > - sentence detection
> > > - token detection
> > > - pos detection
> > > - DiseaseDisorderMention annotation
> > > - negation detection
> > >
> > > My example text is:
> > > I have a disease.
> > > I do not have a disease.
> > > I have some disease.
> > >
> > > "Disease" is an instance of DiseaseDisorderMention.
> > >
> > > However, all my example are tagged to negative:
> > >
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [570-577]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [594-601]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [1099-1106]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > >
> > >
> > > Am I missing something in the pipeline ?
> > >
> > > Thanks,
> > >
> > >
Re: Negation attribute annotator always negative [EXTERNAL]
Posted by Nicolas Paris <ni...@gmail.com>.
Hi Tim
I spotted a bug in my code. I wasn't annotating all my token as
basic_tokens. Right now, this works as expected.
Still, when I use DiseaseDisorderMention, it works as expected. However,
when I am annotating as regular IdentifiedAnnotation, the polarity is
always 0. Maybe this means this is the correct behavior (the
documentation says FocusAnnotationClass = org.apache.ctakes.typesystem.type.textsem.IdentifiedAnnotation )
Le 05 mars 2018 à 23:23, Nicolas Paris écrivait :
> Hi Tim,
>
> I am using ctakes 4.0.0.
>
> The input text is a mimic note, and I added those example sentence
> within the text. They are separated by newlines, and/or other sentences.
>
> I have tested two negation modules: one is a custom ContextAnnotator
> configured as described there: https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+4.0+-+NE+Contexts
> The other is by using the default one:
> /org/apache/ctakes/assertion/models/polarity/sharpi2b2mipacqnegex/model.jar
> (I guess this is also a negation module too ?)
>
> The pipeline is based on :
> - sentence detection -> stanford parser that produces ctakes sentence annotations
> - token detection -> idem but produces ctakes basic token annotations
> - pos detection -> opennlp mayo released within ctakes
> - chunk detection -> opennlp released within ctakes, not sure what it is
>
> Interestingly, when I am annotating the "disease" word with
> "DiseaseDisorderMention", the polarity is always 1.
> But when I am annotating it with "IdentifiedAnnotation" the polarity is
> always 0, withou any modification of the source code but the annotation
> kind.
>
> My guess is I am missing an AE or something.
>
> Le 05 mars 2018 à 21:46, Miller, Timothy écrivait :
> > Hi Nicholas,
> > Good to hear from you! This is quite a surprising result, it is much
> > more typical in my experience for the negation module to miss a
> > negation (because of long distances), so I'd like to try to replicate
> > this. Do you get that result for each sentence at a time? Or only if
> > all three sentences are like that in the note (as in, separated by
> > newlines)?
> > What version of cTAKES? And can you let me know the exact pipeline
> > components? (Which negation annotator, are there any custom modules?)
> > Thanks
> > Tim
> >
> > On Mon, 2018-03-05 at 01:03 +0100, Nicolas Paris wrote:
> > > Hello
> > >
> > > I have integrated the ctakes PolarityCleartkAnalysisEngine in a
> > > pipeline
> > > such:
> > > - sentence detection
> > > - token detection
> > > - pos detection
> > > - DiseaseDisorderMention annotation
> > > - negation detection
> > >
> > > My example text is:
> > > I have a disease.
> > > I do not have a disease.
> > > I have some disease.
> > >
> > > "Disease" is an instance of DiseaseDisorderMention.
> > >
> > > However, all my example are tagged to negative:
> > >
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [570-577]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [594-601]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > > DECODING/EVAL: NEGATED//-1 [1099-1106]
> > > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > >
> > >
> > > Am I missing something in the pipeline ?
> > >
> > > Thanks,
> > >
> > >
Re: Negation attribute annotator always negative [EXTERNAL]
Posted by Nicolas Paris <ni...@gmail.com>.
Hi Tim,
I am using ctakes 4.0.0.
The input text is a mimic note, and I added those example sentence
within the text. They are separated by newlines, and/or other sentences.
I have tested two negation modules: one is a custom ContextAnnotator
configured as described there: https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+4.0+-+NE+Contexts
The other is by using the default one:
/org/apache/ctakes/assertion/models/polarity/sharpi2b2mipacqnegex/model.jar
(I guess this is also a negation module too ?)
The pipeline is based on :
- sentence detection -> stanford parser that produces ctakes sentence annotations
- token detection -> idem but produces ctakes basic token annotations
- pos detection -> opennlp mayo released within ctakes
- chunk detection -> opennlp released within ctakes, not sure what it is
Interestingly, when I am annotating the "disease" word with
"DiseaseDisorderMention", the polarity is always 1.
But when I am annotating it with "IdentifiedAnnotation" the polarity is
always 0, withou any modification of the source code but the annotation
kind.
My guess is I am missing an AE or something.
Le 05 mars 2018 à 21:46, Miller, Timothy écrivait :
> Hi Nicholas,
> Good to hear from you! This is quite a surprising result, it is much
> more typical in my experience for the negation module to miss a
> negation (because of long distances), so I'd like to try to replicate
> this. Do you get that result for each sentence at a time? Or only if
> all three sentences are like that in the note (as in, separated by
> newlines)?
> What version of cTAKES? And can you let me know the exact pipeline
> components? (Which negation annotator, are there any custom modules?)
> Thanks
> Tim
>
> On Mon, 2018-03-05 at 01:03 +0100, Nicolas Paris wrote:
> > Hello
> >
> > I have integrated the ctakes PolarityCleartkAnalysisEngine in a
> > pipeline
> > such:
> > - sentence detection
> > - token detection
> > - pos detection
> > - DiseaseDisorderMention annotation
> > - negation detection
> >
> > My example text is:
> > I have a disease.
> > I do not have a disease.
> > I have some disease.
> >
> > "Disease" is an instance of DiseaseDisorderMention.
> >
> > However, all my example are tagged to negative:
> >
> > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > DECODING/EVAL: NEGATED//-1 [570-577]
> > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > DECODING/EVAL: NEGATED//-1 [594-601]
> > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> > 05 mars 2018 00:51:30 DEBUG AssertionCleartkAnalysisEngine -
> > DECODING/EVAL: NEGATED//-1 [1099-1106]
> > (org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention)
> >
> >
> > Am I missing something in the pipeline ?
> >
> > Thanks,
> >
> >