You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ctakes.apache.org by Natalia Connolly <na...@gmail.com> on 2014/09/30 17:14:53 UTC
Negative polarity - why?
Dear cTAKES Experts,
I have a piece of free text that includes a diagnosis in a stand-alone
sentence, like this:
" - Unspecified pervasive developmental disorder, current or active state -
"
For some reason cTAKES seems to think the polarity of this statement
is negative:
<org.apache.ctakes.assertion.medfacts.types.Concept _indexed="1"
_id="18158" _ref_sofa="3" begin="936" end="980" conceptType="PROBLEM"
conceptText="Unspecified pervasive developmental disorder" externalId="0"
originalEntityExternalId="8563"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="8563" _ref_sofa="3" begin="936" end="980" id="40"
_ref_ontologyConceptArr="8556" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" *polarity="-1"* uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
Why is that?? Can it be the hyphens?
Thanks for any insight,
Natalia Connolly
Re: Negative polarity - why?
Posted by vijay garla <vn...@gmail.com>.
You don't really have any control over how the machine-learning based
negation detection algorithm works. Apparently, that algorithm was trained
on a corpus where '-' represented a negative.
If you use the negex-based negation detection algorithm, you have much more
control over negation detection (you can define your own triggers).
Try ctakes-ytex-uima\desc\analysis_engine\NegexAnnotator.xml instead of the
ctakes assertion analysis engine.
HTH
VJ
On Tue, Oct 7, 2014 at 7:40 PM, Kim Ebert <ki...@perfectsearchcorp.com>
wrote:
> Hi Natalia,
>
> Here are a few variations on your sentence with the results.
>
> The following results in a polarity of 1 for all of the
> DiseaseDisorderMentions. "Unspecified pervasive developmental disorder,
> active state"
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="180" _ref_sofa="1" begin="0" end="44" id="0"
> _ref_ontologyConceptArr="173" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="233" _ref_sofa="1" begin="22" end="44" id="2"
> _ref_ontologyConceptArr="229" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="276" _ref_sofa="1" begin="36" end="44" id="3"
> _ref_ontologyConceptArr="273" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="359" _ref_sofa="1" begin="12" end="44" id="1"
> _ref_ontologyConceptArr="352" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
>
> "Unspecified pervasive developmental disorder,* current or *active state"
> results with one negative polarity.
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="204" _ref_sofa="1" begin="0" end="44" id="0"
> _ref_ontologyConceptArr="197" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="257" _ref_sofa="1" begin="22" end="44" id="2"
> _ref_ontologyConceptArr="253" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="300" _ref_sofa="1" begin="36" end="44" id="3"
> _ref_ontologyConceptArr="297" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="383" _ref_sofa="1" begin="12" end="44" id="1"
> _ref_ontologyConceptArr="376" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
>
>
> "* - *Unspecified pervasive developmental disorder, current or active
> state* - *" results in three negative polarities.
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="180" _ref_sofa="1" begin="39" end="47" id="3"
> _ref_ontologyConceptArr="177" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="true" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="263" _ref_sofa="1" begin="3" end="47" id="0"
> _ref_ontologyConceptArr="256" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="346" _ref_sofa="1" begin="15" end="47" id="1"
> _ref_ontologyConceptArr="339" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="399" _ref_sofa="1" begin="25" end="47" id="2"
> _ref_ontologyConceptArr="395" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
>
> "* - *Unspecified pervasive developmental disorder, current or active
> state" results are the same.
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="172" _ref_sofa="1" begin="39" end="47" id="3"
> _ref_ontologyConceptArr="169" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="255" _ref_sofa="1" begin="3" end="47" id="0"
> _ref_ontologyConceptArr="248" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="338" _ref_sofa="1" begin="15" end="47" id="1"
> _ref_ontologyConceptArr="331" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="391" _ref_sofa="1" begin="25" end="47" id="2"
> _ref_ontologyConceptArr="387" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
>
>
> Oddly, "Unspecified pervasive developmental disorder, current or active
> state -" results in one negative polarity.
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="212" _ref_sofa="1" begin="0" end="44" id="0"
> _ref_ontologyConceptArr="205" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="265" _ref_sofa="1" begin="22" end="44" id="2"
> _ref_ontologyConceptArr="261" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="true" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="308" _ref_sofa="1" begin="36" end="44" id="3"
> _ref_ontologyConceptArr="305" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="true" generic="false" subject="patient" historyOf="0"/>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="391" _ref_sofa="1" begin="12" end="44" id="1"
> _ref_ontologyConceptArr="384" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
> conditional="true" generic="false" subject="patient" historyOf="0"/>
>
> I hope this helps.
>
>
> Kim Ebert1.801.669.7342
> Perfect Search Corphttp://www.perfectsearchcorp.com/
>
> On 09/30/2014 09:14 AM, Natalia Connolly wrote:
>
> Dear cTAKES Experts,
>
> I have a piece of free text that includes a diagnosis in a stand-alone
> sentence, like this:
>
> " - Unspecified pervasive developmental disorder, current or active
> state - "
>
> For some reason cTAKES seems to think the polarity of this
> statement is negative:
>
> <org.apache.ctakes.assertion.medfacts.types.Concept _indexed="1"
> _id="18158" _ref_sofa="3" begin="936" end="980" conceptType="PROBLEM"
> conceptText="Unspecified pervasive developmental disorder" externalId="0"
> originalEntityExternalId="8563"/>
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="8563" _ref_sofa="3" begin="936" end="980" id="40"
> _ref_ontologyConceptArr="8556" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" *polarity="-1"* uncertainty="0"
> conditional="false" generic="false" subject="patient" historyOf="0"/>
>
> Why is that?? Can it be the hyphens?
>
> Thanks for any insight,
>
> Natalia Connolly
>
>
>
Re: Negative polarity - why?
Posted by Kim Ebert <ki...@perfectsearchcorp.com>.
Hi Natalia,
Here are a few variations on your sentence with the results.
The following results in a polarity of 1 for all of the
DiseaseDisorderMentions. "Unspecified pervasive developmental disorder,
active state"
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="180" _ref_sofa="1" begin="0" end="44" id="0"
_ref_ontologyConceptArr="173" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="233" _ref_sofa="1" begin="22" end="44" id="2"
_ref_ontologyConceptArr="229" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="276" _ref_sofa="1" begin="36" end="44" id="3"
_ref_ontologyConceptArr="273" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="359" _ref_sofa="1" begin="12" end="44" id="1"
_ref_ontologyConceptArr="352" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
"Unspecified pervasive developmental disorder,*current or *active state"
results with one negative polarity.
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="204" _ref_sofa="1" begin="0" end="44" id="0"
_ref_ontologyConceptArr="197" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="257" _ref_sofa="1" begin="22" end="44" id="2"
_ref_ontologyConceptArr="253" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="300" _ref_sofa="1" begin="36" end="44" id="3"
_ref_ontologyConceptArr="297" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="383" _ref_sofa="1" begin="12" end="44" id="1"
_ref_ontologyConceptArr="376" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
"*- *Unspecified pervasive developmental disorder, current or active
state*- *" results in three negative polarities.
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="180" _ref_sofa="1" begin="39" end="47" id="3"
_ref_ontologyConceptArr="177" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="true" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="263" _ref_sofa="1" begin="3" end="47" id="0"
_ref_ontologyConceptArr="256" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="346" _ref_sofa="1" begin="15" end="47" id="1"
_ref_ontologyConceptArr="339" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="399" _ref_sofa="1" begin="25" end="47" id="2"
_ref_ontologyConceptArr="395" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
"*- *Unspecified pervasive developmental disorder, current or active
state" results are the same.
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="172" _ref_sofa="1" begin="39" end="47" id="3"
_ref_ontologyConceptArr="169" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="1"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="255" _ref_sofa="1" begin="3" end="47" id="0"
_ref_ontologyConceptArr="248" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="338" _ref_sofa="1" begin="15" end="47" id="1"
_ref_ontologyConceptArr="331" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="391" _ref_sofa="1" begin="25" end="47" id="2"
_ref_ontologyConceptArr="387" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
Oddly, "Unspecified pervasive developmental disorder, current or active
state -" results in one negative polarity.
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="212" _ref_sofa="1" begin="0" end="44" id="0"
_ref_ontologyConceptArr="205" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="*-1*" uncertainty="0"
conditional="false" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="265" _ref_sofa="1" begin="22" end="44" id="2"
_ref_ontologyConceptArr="261" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="true" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="308" _ref_sofa="1" begin="36" end="44" id="3"
_ref_ontologyConceptArr="305" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="true" generic="false" subject="patient" historyOf="0"/>
<org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
_indexed="1" _id="391" _ref_sofa="1" begin="12" end="44" id="1"
_ref_ontologyConceptArr="384" typeID="2" segmentID="SIMPLE_SEGMENT"
discoveryTechnique="1" confidence="1.0" polarity="1" uncertainty="0"
conditional="true" generic="false" subject="patient" historyOf="0"/>
I hope this helps.
Kim Ebert
1.801.669.7342
Perfect Search Corp
http://www.perfectsearchcorp.com/
On 09/30/2014 09:14 AM, Natalia Connolly wrote:
> Dear cTAKES Experts,
>
> I have a piece of free text that includes a diagnosis in a
> stand-alone sentence, like this:
>
> " - Unspecified pervasive developmental disorder, current or active
> state - "
>
> For some reason cTAKES seems to think the polarity of this
> statement is negative:
>
> <org.apache.ctakes.assertion.medfacts.types.Concept _indexed="1"
> _id="18158" _ref_sofa="3" begin="936" end="980" conceptType="PROBLEM"
> conceptText="Unspecified pervasive developmental disorder"
> externalId="0" originalEntityExternalId="8563"/>
>
> <org.apache.ctakes.typesystem.type.textsem.DiseaseDisorderMention
> _indexed="1" _id="8563" _ref_sofa="3" begin="936" end="980" id="40"
> _ref_ontologyConceptArr="8556" typeID="2" segmentID="SIMPLE_SEGMENT"
> discoveryTechnique="1" confidence="1.0" *polarity="-1"*
> uncertainty="0" conditional="false" generic="false" subject="patient"
> historyOf="0"/>
>
> Why is that?? Can it be the hyphens?
>
> Thanks for any insight,
>
> Natalia Connolly
>