You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by Ab...@cognizant.com on 2018/02/07 11:46:51 UTC
Lab Value - Range finder
Hi All,
I am trying to extend the current Lab Value finder annotator to include the Range annotation also.
"Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
Using current module , I am able to pull lab value as " BUN 10 mg" from above text and trying to tag "8-23" as reference range.
final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap = createCoveringMap( jCas, valueClasses,
Arrays.asList( FractionAnnotation.class, RangeAnnotation.class, MeasurementAnnotation.class ) );
The range annotator is already added in the Labvalue finder class, but only MeasurementAnnotation is getting tagged in the output. Can anyone help here?
Thanks,
Abilash Mathew
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
Fwd: Lab Value - Range finder
Posted by Kean Kaufmann <ke...@recordsone.com>.
Oops, forgot to cc: dev. Happy Monday...
---------- Forwarded message ----------
From: Kean Kaufmann <ke...@recordsone.com>
Date: Mon, Feb 12, 2018 at 10:14 AM
Subject: Re: Lab Value - Range finder
To: Abilash.Mathew@cognizant.com
> I could see setReferenceRangeNarrative method in LabMention class. Is
that the one are you referring?
Yes -- that's the setter method generated from the type system:
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-
system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Feature: referenceRangeNarrative
Value type: org.apache.ctakes.typesystem.type.textsem.
LabReferenceRangeModifier
> Also, what is your suggestion to create new class similar to
LabValueFinder?
Well, if the LabValueFinder suits your needs, then I'd suggest using its
annotations as starting points.
Run the LabValueFinder and inspect the LabMentions it creates.
If getLabValue() gives you a RangeAnnotation, it's probably the reference
range,
which the LabValueFinder fell back to because it couldn't find a value.
Otherwise, search for RangeAnnotations nearby.
You may find various org.apache.uima.fit.util.JCasUtil
<https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/JCasUtil.html>
methods
useful, e.g. selectBetween, selectPreceding, selectFollowing.
In my data, the range could often be found between the LabMention and the
LabValue; your data may vary.
For a general-purpose annotator, you'd want to make that configurable.
Also, make you're getting the RangeAnnotations you expect.
Last year, I found it expedient to write a quick-fix regular-expression
annotator to pick up decimal ranges like the ones below.
It would be more civic-minded to change the annotator that generates
RangeAnnotations
(the ContextDependentTokenizerAnnotator? Not sure offhand).
"Potassium Latest Range: 3.5-5.3 mmol/L 3.8\n" + // range not annotated
> "TSH, High Sensitivity Latest Range: 0.450-5.100 uIU/mL 1.939\n" + //
> range not annotated
On Mon, Feb 12, 2018 at 6:24 AM, <Ab...@cognizant.com> wrote:
> Thanks Kean for the suggestions. I could see setReferenceRangeNarrative
> method in LabMention class. Is that the one are you referring? Also, what
> is your suggestion to create new class similar to LabValueFinder?
>
> Regards,
> Abilash Mathew
> -----Original Message-----
> From: Kean Kaufmann [mailto:kean@recordsone.com]
> Sent: Friday, February 9, 2018 10:35 PM
> To: dev@ctakes.apache.org
> Subject: Re: Lab Value - Range finder
>
> Hi Abilash,
>
> By design, the Lab Value annotator avoids ranges if possible:
>
> https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/sr
> c/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
>
> // prefer non-range values, if any
> > value = candidateList.stream()
> > .filter( a -> !(a instanceof RangeAnnotation) )
> > .findFirst()
> > .orElse( candidateList.get( 0 ) );
>
>
> The design is intended to cope with the widest possible variety of
> formats, and avoid confusion.
>
> If you know where the reference ranges are going to be in your data, I'd
> suggest annotating those separately, using the referenceRangeNarrative
> feature with LabReferenceRange values rather than conflating them with
> labValue / LabMention.
>
> https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-sy
> stem/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
>
> Caveat: When I wrote this last year, the range annotator didn't seem to be
> picking up on decimal points. So, below, the first two got
> RangeAnnotations, and the second two didn't.
>
> "Sodium Latest Range: 135-145 mmol/L 138\n" +
> > "Anion Gap Latest Range: 13-16 mmol/L\n" + "Potassium Latest Range:
> > 3.5-5.3 mmol/L 3.8\n" + // range not annotated "TSH, High Sensitivity
> > Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
>
>
>
>
> On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com> wrote:
>
> > Hi All,
> >
> > I am trying to extend the current Lab Value finder annotator to
> > include the Range annotation also.
> >
> > "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
> >
> > Using current module , I am able to pull lab value as " BUN 10 mg"
> > from above text and trying to tag "8-23" as reference range.
> >
> > final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> > createCoveringMap( jCas, valueClasses,
> > Arrays.asList( FractionAnnotation.class,
> > RangeAnnotation.class, MeasurementAnnotation.class ) );
> >
> > The range annotator is already added in the Labvalue finder class, but
> > only MeasurementAnnotation is getting tagged in the output. Can anyone
> > help here?
> >
> >
> > Thanks,
> > Abilash Mathew
> > This e-mail and any files transmitted with it are for the sole use of
> > the intended recipient(s) and may contain confidential and privileged
> > information. If you are not the intended recipient(s), please reply to
> > the sender and destroy all copies of the original message. Any
> > unauthorized review, use, disclosure, dissemination, forwarding,
> > printing or copying of this email, and/or any action taken in reliance
> > on the contents of this e-mail is strictly prohibited and may be
> > unlawful. Where permitted by applicable law, this e-mail and other
> > e-mail communications sent to and from Cognizant e-mail addresses may be
> monitored.
> >
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>
RE: Lab Value - Range finder
Posted by Ab...@cognizant.com.
Thanks Kean. I was able to get the desired output using selectCovered method.
From: Kean Kaufmann [mailto:kean@recordsone.com]
Sent: Friday, March 2, 2018 12:30 AM
To: Mathew, Abilash (Cognizant) <Ab...@cognizant.com>
Cc: dev@ctakes.apache.org
Subject: Re: Lab Value - Range finder
Abilash,
The org.apache.uima.fit.util.JCasUtil methods selectCovered and indexCovered may be what you're looking for here.
If you run cTAKES on the following text:
"Sodium Latest Range: 135-145 mmol/L 138"
note that "135-145" is covered by a RangeAnnotation;
"135" and "145" are covered by NumTokens;
and "-" is covered by a PunctuationToken.
Similarly, "21.1 L" is covered by a MeasurementAnnotation, in which "21.1" is covered by both a FractionAnnotation and a NumToken, and "L" by a WordToken.
Have you used the CAS Visual Debugger? https://uima.apache.org/d/uimaj-current/tools.html#ugr.tools.cvd
It will show you all the annotations covering a given stretch of text -- very helpful.
On Wed, Feb 28, 2018 at 6:51 AM, <Ab...@cognizant.com>> wrote:
Kean,
I was able to get the annotation tagged as Range and measurement annotations as shown below. As a next step, I would like to get the individual token like 42 ,52 and 21 etc. I know I can try a pattern or index based search to extract the information, but I am trying to see if any options currently available in CTAKES to do the same.
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: LabMention(349-352): HCT
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: RangeAnnotation(365-370): 42-52
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: MeasurementAnnotation(354-360): 21.1 L
Thanks,
Abilash Mathew
From: Kean Kaufmann [mailto:kean@recordsone.com<ma...@recordsone.com>]
Sent: Monday, February 12, 2018 8:44 PM
To: Mathew, Abilash (Cognizant) <Ab...@cognizant.com>>
Subject: Re: Lab Value - Range finder
> I could see setReferenceRangeNarrative method in LabMention class. Is that the one are you referring?
Yes -- that's the setter method generated from the type system: https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Feature: referenceRangeNarrative
Value type: org.apache.ctakes.typesystem.type.textsem.LabReferenceRangeModifier
> Also, what is your suggestion to create new class similar to LabValueFinder?
Well, if the LabValueFinder suits your needs, then I'd suggest using its annotations as starting points.
Run the LabValueFinder and inspect the LabMentions it creates.
If getLabValue() gives you a RangeAnnotation, it's probably the reference range,
which the LabValueFinder fell back to because it couldn't find a value.
Otherwise, search for RangeAnnotations nearby.
You may find various org.apache.uima.fit.util.JCasUtil<https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/JCasUtil.html> methods useful, e.g. selectBetween, selectPreceding, selectFollowing.
In my data, the range could often be found between the LabMention and the LabValue; your data may vary.
For a general-purpose annotator, you'd want to make that configurable.
Also, make you're getting the RangeAnnotations you expect.
Last year, I found it expedient to write a quick-fix regular-expression annotator to pick up decimal ranges like the ones below.
It would be more civic-minded to change the annotator that generates RangeAnnotations
(the ContextDependentTokenizerAnnotator? Not sure offhand).
"Potassium Latest Range: 3.5-5.3 mmol/L 3.8\n" + // range not annotated
"TSH, High Sensitivity Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
On Mon, Feb 12, 2018 at 6:24 AM, <Ab...@cognizant.com>> wrote:
Thanks Kean for the suggestions. I could see setReferenceRangeNarrative method in LabMention class. Is that the one are you referring? Also, what is your suggestion to create new class similar to LabValueFinder?
Regards,
Abilash Mathew
-----Original Message-----
From: Kean Kaufmann [mailto:kean@recordsone.com<ma...@recordsone.com>]
Sent: Friday, February 9, 2018 10:35 PM
To: dev@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: Lab Value - Range finder
Hi Abilash,
By design, the Lab Value annotator avoids ranges if possible:
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
// prefer non-range values, if any
> value = candidateList.stream()
> .filter( a -> !(a instanceof RangeAnnotation) )
> .findFirst()
> .orElse( candidateList.get( 0 ) );
The design is intended to cope with the widest possible variety of formats, and avoid confusion.
If you know where the reference ranges are going to be in your data, I'd suggest annotating those separately, using the referenceRangeNarrative feature with LabReferenceRange values rather than conflating them with labValue / LabMention.
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Caveat: When I wrote this last year, the range annotator didn't seem to be picking up on decimal points. So, below, the first two got RangeAnnotations, and the second two didn't.
"Sodium Latest Range: 135-145 mmol/L 138\n" +
> "Anion Gap Latest Range: 13-16 mmol/L\n" + "Potassium Latest Range:
> 3.5-5.3 mmol/L 3.8\n" + // range not annotated "TSH, High Sensitivity
> Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com>> wrote:
> Hi All,
>
> I am trying to extend the current Lab Value finder annotator to
> include the Range annotation also.
>
> "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
>
> Using current module , I am able to pull lab value as " BUN 10 mg"
> from above text and trying to tag "8-23" as reference range.
>
> final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> createCoveringMap( jCas, valueClasses,
> Arrays.asList( FractionAnnotation.class,
> RangeAnnotation.class, MeasurementAnnotation.class ) );
>
> The range annotator is already added in the Labvalue finder class, but
> only MeasurementAnnotation is getting tagged in the output. Can anyone
> help here?
>
>
> Thanks,
> Abilash Mathew
> This e-mail and any files transmitted with it are for the sole use of
> the intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to
> the sender and destroy all copies of the original message. Any
> unauthorized review, use, disclosure, dissemination, forwarding,
> printing or copying of this email, and/or any action taken in reliance
> on the contents of this e-mail is strictly prohibited and may be
> unlawful. Where permitted by applicable law, this e-mail and other
> e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
>
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
Re: Lab Value - Range finder
Posted by Kean Kaufmann <ke...@recordsone.com>.
Abilash,
The org.apache.uima.fit.util.JCasUtil methods selectCovered and
indexCovered may be what you're looking for here.
If you run cTAKES on the following text:
"Sodium Latest Range: 135-145 mmol/L 138"
note that "135-145" is covered by a RangeAnnotation;
"135" and "145" are covered by NumTokens;
and "-" is covered by a PunctuationToken.
Similarly, "21.1 L" is covered by a MeasurementAnnotation, in which "21.1"
is covered by both a FractionAnnotation and a NumToken, and "L" by a
WordToken.
Have you used the CAS Visual Debugger?
https://uima.apache.org/d/uimaj-current/tools.html#ugr.tools.cvd
It will show you all the annotations covering a given stretch of text --
very helpful.
On Wed, Feb 28, 2018 at 6:51 AM, <Ab...@cognizant.com> wrote:
> Kean,
>
> I was able to get the annotation tagged as Range and measurement
> annotations as shown below. As a next step, I would like to get the
> individual token like 42 ,52 and 21 etc. I know I can try a pattern or
> index based search to extract the information, but I am trying to see if
> any options currently available in CTAKES to do the same.
>
> 28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value:
> LabMention(349-352): HCT
>
> 28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value:
> RangeAnnotation(365-370): 42-52
>
> 28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value:
> MeasurementAnnotation(354-360): 21.1 L
>
>
>
> Thanks,
>
> Abilash Mathew
>
> *From:* Kean Kaufmann [mailto:kean@recordsone.com]
> *Sent:* Monday, February 12, 2018 8:44 PM
> *To:* Mathew, Abilash (Cognizant) <Ab...@cognizant.com>
> *Subject:* Re: Lab Value - Range finder
>
>
>
> > I could see setReferenceRangeNarrative method in LabMention class. Is
> that the one are you referring?
>
>
>
> Yes -- that's the setter method generated from the type system:
> https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-
> system/src/main/resources/org/apache/ctakes/typesystem/
> types/TypeSystem.xml
>
> Feature: referenceRangeNarrative
>
> Value type: org.apache.ctakes.typesystem.type.textsem.
> LabReferenceRangeModifier
>
>
>
> > Also, what is your suggestion to create new class similar to
> LabValueFinder?
>
>
>
> Well, if the LabValueFinder suits your needs, then I'd suggest using its
> annotations as starting points.
>
> Run the LabValueFinder and inspect the LabMentions it creates.
>
> If getLabValue() gives you a RangeAnnotation, it's probably the reference
> range,
>
> which the LabValueFinder fell back to because it couldn't find a value.
>
> Otherwise, search for RangeAnnotations nearby.
>
> You may find various org.apache.uima.fit.util.JCasUtil
> <https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/JCasUtil.html> methods
> useful, e.g. selectBetween, selectPreceding, selectFollowing.
>
> In my data, the range could often be found between the LabMention and the
> LabValue; your data may vary.
>
> For a general-purpose annotator, you'd want to make that configurable.
>
>
>
> Also, make you're getting the RangeAnnotations you expect.
>
> Last year, I found it expedient to write a quick-fix regular-expression
> annotator to pick up decimal ranges like the ones below.
>
> It would be more civic-minded to change the annotator that generates
> RangeAnnotations
>
> (the ContextDependentTokenizerAnnotator? Not sure offhand).
>
>
>
> "Potassium Latest Range:
> 3.5-5.3 mmol/L 3.8\n" + // range not annotated
> "TSH, High Sensitivity Latest
> Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
>
>
>
>
>
> On Mon, Feb 12, 2018 at 6:24 AM, <Ab...@cognizant.com> wrote:
>
> Thanks Kean for the suggestions. I could see setReferenceRangeNarrative
> method in LabMention class. Is that the one are you referring? Also, what
> is your suggestion to create new class similar to LabValueFinder?
>
> Regards,
> Abilash Mathew
>
> -----Original Message-----
> From: Kean Kaufmann [mailto:kean@recordsone.com]
> Sent: Friday, February 9, 2018 10:35 PM
> To: dev@ctakes.apache.org
> Subject: Re: Lab Value - Range finder
>
> Hi Abilash,
>
> By design, the Lab Value annotator avoids ranges if possible:
>
> https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/
> src/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
>
> // prefer non-range values, if any
> > value = candidateList.stream()
> > .filter( a -> !(a instanceof RangeAnnotation) )
> > .findFirst()
> > .orElse( candidateList.get( 0 ) );
>
>
> The design is intended to cope with the widest possible variety of
> formats, and avoid confusion.
>
> If you know where the reference ranges are going to be in your data, I'd
> suggest annotating those separately, using the referenceRangeNarrative
> feature with LabReferenceRange values rather than conflating them with
> labValue / LabMention.
>
> https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-
> system/src/main/resources/org/apache/ctakes/typesystem/
> types/TypeSystem.xml
>
> Caveat: When I wrote this last year, the range annotator didn't seem to be
> picking up on decimal points. So, below, the first two got
> RangeAnnotations, and the second two didn't.
>
> "Sodium Latest Range: 135-145 mmol/L 138\n" +
> > "Anion Gap Latest Range: 13-16 mmol/L\n" + "Potassium Latest Range:
> > 3.5-5.3 mmol/L 3.8\n" + // range not annotated "TSH, High Sensitivity
> > Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
>
>
>
>
> On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com> wrote:
>
> > Hi All,
> >
> > I am trying to extend the current Lab Value finder annotator to
> > include the Range annotation also.
> >
> > "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
> >
> > Using current module , I am able to pull lab value as " BUN 10 mg"
> > from above text and trying to tag "8-23" as reference range.
> >
> > final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> > createCoveringMap( jCas, valueClasses,
> > Arrays.asList( FractionAnnotation.class,
> > RangeAnnotation.class, MeasurementAnnotation.class ) );
> >
> > The range annotator is already added in the Labvalue finder class, but
> > only MeasurementAnnotation is getting tagged in the output. Can anyone
> > help here?
> >
> >
> > Thanks,
> > Abilash Mathew
> > This e-mail and any files transmitted with it are for the sole use of
> > the intended recipient(s) and may contain confidential and privileged
> > information. If you are not the intended recipient(s), please reply to
> > the sender and destroy all copies of the original message. Any
> > unauthorized review, use, disclosure, dissemination, forwarding,
> > printing or copying of this email, and/or any action taken in reliance
> > on the contents of this e-mail is strictly prohibited and may be
> > unlawful. Where permitted by applicable law, this e-mail and other
> > e-mail communications sent to and from Cognizant e-mail addresses may be
> monitored.
> >
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>
RE: Lab Value - Range finder
Posted by Ab...@cognizant.com.
Kean,
I was able to get the annotation tagged as Range and measurement annotations as shown below. As a next step, I would like to get the individual token like 42 ,52 and 21 etc. I know I can try a pattern or index based search to extract the information, but I am trying to see if any options currently available in CTAKES to do the same.
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: LabMention(349-352): HCT
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: RangeAnnotation(365-370): 42-52
28 Feb 2018 15:04:04 INFO LabValueFinder - Set to value: MeasurementAnnotation(354-360): 21.1 L
Thanks,
Abilash Mathew
From: Kean Kaufmann [mailto:kean@recordsone.com]
Sent: Monday, February 12, 2018 8:44 PM
To: Mathew, Abilash (Cognizant) <Ab...@cognizant.com>
Subject: Re: Lab Value - Range finder
> I could see setReferenceRangeNarrative method in LabMention class. Is that the one are you referring?
Yes -- that's the setter method generated from the type system: https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Feature: referenceRangeNarrative
Value type: org.apache.ctakes.typesystem.type.textsem.LabReferenceRangeModifier
> Also, what is your suggestion to create new class similar to LabValueFinder?
Well, if the LabValueFinder suits your needs, then I'd suggest using its annotations as starting points.
Run the LabValueFinder and inspect the LabMentions it creates.
If getLabValue() gives you a RangeAnnotation, it's probably the reference range,
which the LabValueFinder fell back to because it couldn't find a value.
Otherwise, search for RangeAnnotations nearby.
You may find various org.apache.uima.fit.util.JCasUtil<https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/JCasUtil.html> methods useful, e.g. selectBetween, selectPreceding, selectFollowing.
In my data, the range could often be found between the LabMention and the LabValue; your data may vary.
For a general-purpose annotator, you'd want to make that configurable.
Also, make you're getting the RangeAnnotations you expect.
Last year, I found it expedient to write a quick-fix regular-expression annotator to pick up decimal ranges like the ones below.
It would be more civic-minded to change the annotator that generates RangeAnnotations
(the ContextDependentTokenizerAnnotator? Not sure offhand).
"Potassium Latest Range: 3.5-5.3 mmol/L 3.8\n" + // range not annotated
"TSH, High Sensitivity Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
On Mon, Feb 12, 2018 at 6:24 AM, <Ab...@cognizant.com>> wrote:
Thanks Kean for the suggestions. I could see setReferenceRangeNarrative method in LabMention class. Is that the one are you referring? Also, what is your suggestion to create new class similar to LabValueFinder?
Regards,
Abilash Mathew
-----Original Message-----
From: Kean Kaufmann [mailto:kean@recordsone.com<ma...@recordsone.com>]
Sent: Friday, February 9, 2018 10:35 PM
To: dev@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: Lab Value - Range finder
Hi Abilash,
By design, the Lab Value annotator avoids ranges if possible:
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
// prefer non-range values, if any
> value = candidateList.stream()
> .filter( a -> !(a instanceof RangeAnnotation) )
> .findFirst()
> .orElse( candidateList.get( 0 ) );
The design is intended to cope with the widest possible variety of formats, and avoid confusion.
If you know where the reference ranges are going to be in your data, I'd suggest annotating those separately, using the referenceRangeNarrative feature with LabReferenceRange values rather than conflating them with labValue / LabMention.
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Caveat: When I wrote this last year, the range annotator didn't seem to be picking up on decimal points. So, below, the first two got RangeAnnotations, and the second two didn't.
"Sodium Latest Range: 135-145 mmol/L 138\n" +
> "Anion Gap Latest Range: 13-16 mmol/L\n" + "Potassium Latest Range:
> 3.5-5.3 mmol/L 3.8\n" + // range not annotated "TSH, High Sensitivity
> Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com>> wrote:
> Hi All,
>
> I am trying to extend the current Lab Value finder annotator to
> include the Range annotation also.
>
> "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
>
> Using current module , I am able to pull lab value as " BUN 10 mg"
> from above text and trying to tag "8-23" as reference range.
>
> final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> createCoveringMap( jCas, valueClasses,
> Arrays.asList( FractionAnnotation.class,
> RangeAnnotation.class, MeasurementAnnotation.class ) );
>
> The range annotator is already added in the Labvalue finder class, but
> only MeasurementAnnotation is getting tagged in the output. Can anyone
> help here?
>
>
> Thanks,
> Abilash Mathew
> This e-mail and any files transmitted with it are for the sole use of
> the intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to
> the sender and destroy all copies of the original message. Any
> unauthorized review, use, disclosure, dissemination, forwarding,
> printing or copying of this email, and/or any action taken in reliance
> on the contents of this e-mail is strictly prohibited and may be
> unlawful. Where permitted by applicable law, this e-mail and other
> e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
>
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
RE: Lab Value - Range finder
Posted by Ab...@cognizant.com.
Thanks Kean for the suggestions. I could see setReferenceRangeNarrative method in LabMention class. Is that the one are you referring? Also, what is your suggestion to create new class similar to LabValueFinder?
Regards,
Abilash Mathew
-----Original Message-----
From: Kean Kaufmann [mailto:kean@recordsone.com]
Sent: Friday, February 9, 2018 10:35 PM
To: dev@ctakes.apache.org
Subject: Re: Lab Value - Range finder
Hi Abilash,
By design, the Lab Value annotator avoids ranges if possible:
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
// prefer non-range values, if any
> value = candidateList.stream()
> .filter( a -> !(a instanceof RangeAnnotation) )
> .findFirst()
> .orElse( candidateList.get( 0 ) );
The design is intended to cope with the widest possible variety of formats, and avoid confusion.
If you know where the reference ranges are going to be in your data, I'd suggest annotating those separately, using the referenceRangeNarrative feature with LabReferenceRange values rather than conflating them with labValue / LabMention.
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Caveat: When I wrote this last year, the range annotator didn't seem to be picking up on decimal points. So, below, the first two got RangeAnnotations, and the second two didn't.
"Sodium Latest Range: 135-145 mmol/L 138\n" +
> "Anion Gap Latest Range: 13-16 mmol/L\n" + "Potassium Latest Range:
> 3.5-5.3 mmol/L 3.8\n" + // range not annotated "TSH, High Sensitivity
> Latest Range: 0.450-5.100 uIU/mL 1.939\n" + // range not annotated
On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com> wrote:
> Hi All,
>
> I am trying to extend the current Lab Value finder annotator to
> include the Range annotation also.
>
> "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
>
> Using current module , I am able to pull lab value as " BUN 10 mg"
> from above text and trying to tag "8-23" as reference range.
>
> final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> createCoveringMap( jCas, valueClasses,
> Arrays.asList( FractionAnnotation.class,
> RangeAnnotation.class, MeasurementAnnotation.class ) );
>
> The range annotator is already added in the Labvalue finder class, but
> only MeasurementAnnotation is getting tagged in the output. Can anyone
> help here?
>
>
> Thanks,
> Abilash Mathew
> This e-mail and any files transmitted with it are for the sole use of
> the intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to
> the sender and destroy all copies of the original message. Any
> unauthorized review, use, disclosure, dissemination, forwarding,
> printing or copying of this email, and/or any action taken in reliance
> on the contents of this e-mail is strictly prohibited and may be
> unlawful. Where permitted by applicable law, this e-mail and other
> e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
>
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
Re: Lab Value - Range finder
Posted by Kean Kaufmann <ke...@recordsone.com>.
Hi Abilash,
By design, the Lab Value annotator avoids ranges if possible:
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/LabValueFinder.java
// prefer non-range values, if any
> value = candidateList.stream()
> .filter( a -> !(a instanceof RangeAnnotation) )
> .findFirst()
> .orElse( candidateList.get( 0 ) );
The design is intended to cope with the widest possible variety of formats,
and avoid confusion.
If you know where the reference ranges are going to be in your data, I'd
suggest annotating those separately,
using the referenceRangeNarrative feature with LabReferenceRange values
rather than conflating them with labValue / LabMention.
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-type-system/src/main/resources/org/apache/ctakes/typesystem/types/TypeSystem.xml
Caveat: When I wrote this last year, the range annotator didn't seem to be
picking up on decimal points. So, below, the first two got
RangeAnnotations, and the second two didn't.
"Sodium Latest Range: 135-145 mmol/L 138\n" +
> "Anion Gap Latest Range: 13-16 mmol/L\n" +
> "Potassium Latest Range: 3.5-5.3 mmol/L 3.8\n" + // range not annotated
> "TSH, High Sensitivity Latest Range: 0.450-5.100 uIU/mL 1.939\n" + //
> range not annotated
On Wed, Feb 7, 2018 at 6:46 AM, <Ab...@cognizant.com> wrote:
> Hi All,
>
> I am trying to extend the current Lab Value finder annotator to include
> the Range annotation also.
>
> "Blood Urea Nitrogen (BUN) 10 mg/dL 8-23"
>
> Using current module , I am able to pull lab value as " BUN 10 mg" from
> above text and trying to tag "8-23" as reference range.
>
> final Map<Annotation, List<IdentifiedAnnotation>> subsumeMap =
> createCoveringMap( jCas, valueClasses,
> Arrays.asList( FractionAnnotation.class,
> RangeAnnotation.class, MeasurementAnnotation.class ) );
>
> The range annotator is already added in the Labvalue finder class, but
> only MeasurementAnnotation is getting tagged in the output. Can anyone help
> here?
>
>
> Thanks,
> Abilash Mathew
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>