You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by "Miller, Timothy" <Ti...@childrens.harvard.edu> on 2021/12/07 17:21:55 UTC
Re: empty preferredText [EXTERNAL]
OK, I thought this might be what's happening. I did check my 2021 UMLS release and the cui does seem to have a preferred text but I think my container is using an older release. For what it's worth the CUI is:
C0360554
and a sentence that reproduces the issue in CVD with the current release is:
'Patient had problems tolerating oral hydrocortisone.'
I will see if I can find the older UMLS release lying around. I think the right workaround for now is your suggestion of using the covered text.
Tim
On Tue, 2021-12-07 at 17:59 +0100, Peter Abramowitsch wrote:
* External Email - Caution *
Hi Tim,
Yes, I've definitely encountered it. It happens when the concept has a
CUI_TERM which has matched the text, but there is no corresponding entry in
the SNOMED or other vocab table mapping CUI to SNOMED. The obvious choice
is to use the covered text as a surrogate, but technically it could be PHI
if that matters to you. The other thing is to see if there's an MSH term
that maps using the metathesaurus. If so, including MSH in your dictionary
as a src AND dest vocab will solve the problem.
Peter
On Tue, Dec 7, 2021 at 5:45 PM Miller, Timothy <
<ma...@childrens.harvard.edu>
Timothy.Miller@childrens.harvard.edu
> wrote:
Hello,
I'm using the dictionary lookup (through ctakes-web-rest) and trying to
read off the preferredText that comes back as a human-readable way to
display the CUI. On a very small percentage, there does not seem to be any
preferredText. Has anyone else encountered this? Is this a limitation of
the underlying ontologies or a bug we can address?
Tim
Re: empty preferredText [EXTERNAL]
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
Hear, hear!
________________________________________
From: Peter Abramowitsch <pa...@gmail.com>
Sent: Tuesday, December 7, 2021 1:17 PM
To: dev@ctakes.apache.org
Subject: Re: empty preferredText [EXTERNAL]
* External Email - Caution *
"but I might revisit it on a snowy afternoon this winter. " >> let's hope
for lots of those and good holidays for all.
Peter
On Tue, Dec 7, 2021 at 7:05 PM Finan, Sean <Se...@childrens.harvard.edu>
wrote:
> I think that you are both correct.
>
> One or more preferred texts for a concept should be available if the user
> created a local copy of the umls that contains one a vocabulary that
> contains a preferred text. This is the first step in the process of
> creating a ctakes dictionary and takes place before the ctakes dictionary
> creator gui is started. I forget what the umls tool is called.
> Metamorphosys? Using default settings in metamorphosys(?) should make
> available a lot of preferred texts.
>
> I think that the ctakes dictionary creator does need to have such
> vocabularies checked as src and dest. I think that the original intention
> of separate src and dest options was so that you could obtain information
> like synonyms and preferred text from a vocabulary without writing a column
> of its codes to the dictionary. It may not be working as intended or I may
> have deviated from that tactic at some point.
>
> A checked dest vocabulary will add a column to the dictionary database,
> but you can accelerate searches in ctakes by excluding them as desired code
> sources. I am probably diving too far into the weeds with that
> information.
>
> While it is possible that the ctakes dictionary creator leaves unspecified
> preferred texts unavailable for a 'good' reason, it is more likely that I
> wasn't paying enough attention during implementation. It has been 2 years
> since I looked at the dictionary creator (for the case-sensitive lookup),
> but I might revisit it on a snowy afternoon this winter. Of course, if
> anybody out there in the dev world would like to take a first crack at it
> ...
>
> Sean
> ________________________________________
> From: Peter Abramowitsch <pa...@gmail.com>
> Sent: Tuesday, December 7, 2021 12:34 PM
> To: dev@ctakes.apache.org
> Subject: Re: empty preferredText [EXTERNAL]
>
> * External Email - Caution *
>
>
> I think the issue is that preferred text in the dictionary is only
> populated by matches from the "dest" vocabularies and it uses *their*
> preferred text. If there's no match in any of them, then it should put the
> CUI's own preferred text entry in the dictionary, but it doesn't. I'm
> pretty sure It's available during the dictionary creation process, but
> probably not used.
>
> On Tue, Dec 7, 2021 at 6:22 PM Miller, Timothy <
> Timothy.Miller@childrens.harvard.edu> wrote:
>
> > OK, I thought this might be what's happening. I did check my 2021 UMLS
> > release and the cui does seem to have a preferred text but I think my
> > container is using an older release. For what it's worth the CUI is:
> > C0360554
> >
> > and a sentence that reproduces the issue in CVD with the current release
> > is:
> >
> > 'Patient had problems tolerating oral hydrocortisone.'
> >
> > I will see if I can find the older UMLS release lying around. I think the
> > right workaround for now is your suggestion of using the covered text.
> >
> > Tim
> >
> >
> > On Tue, 2021-12-07 at 17:59 +0100, Peter Abramowitsch wrote:
> >
> > * External Email - Caution *
> >
> >
> >
> > Hi Tim,
> >
> >
> > Yes, I've definitely encountered it. It happens when the concept has a
> >
> > CUI_TERM which has matched the text, but there is no corresponding entry
> in
> >
> > the SNOMED or other vocab table mapping CUI to SNOMED. The obvious
> choice
> >
> > is to use the covered text as a surrogate, but technically it could be
> PHI
> >
> > if that matters to you. The other thing is to see if there's an MSH term
> >
> > that maps using the metathesaurus. If so, including MSH in your
> dictionary
> >
> > as a src AND dest vocab will solve the problem.
> >
> >
> > Peter
> >
> >
> >
> > On Tue, Dec 7, 2021 at 5:45 PM Miller, Timothy <
> >
> > <ma...@childrens.harvard.edu>
> >
> > Timothy.Miller@childrens.harvard.edu
> >
> > > wrote:
> >
> >
> > Hello,
> >
> > I'm using the dictionary lookup (through ctakes-web-rest) and trying to
> >
> > read off the preferredText that comes back as a human-readable way to
> >
> > display the CUI. On a very small percentage, there does not seem to be
> any
> >
> > preferredText. Has anyone else encountered this? Is this a limitation of
> >
> > the underlying ontologies or a bug we can address?
> >
> > Tim
> >
> >
> >
>
Re: empty preferredText [EXTERNAL]
Posted by Peter Abramowitsch <pa...@gmail.com>.
"but I might revisit it on a snowy afternoon this winter. " >> let's hope
for lots of those and good holidays for all.
Peter
On Tue, Dec 7, 2021 at 7:05 PM Finan, Sean <Se...@childrens.harvard.edu>
wrote:
> I think that you are both correct.
>
> One or more preferred texts for a concept should be available if the user
> created a local copy of the umls that contains one a vocabulary that
> contains a preferred text. This is the first step in the process of
> creating a ctakes dictionary and takes place before the ctakes dictionary
> creator gui is started. I forget what the umls tool is called.
> Metamorphosys? Using default settings in metamorphosys(?) should make
> available a lot of preferred texts.
>
> I think that the ctakes dictionary creator does need to have such
> vocabularies checked as src and dest. I think that the original intention
> of separate src and dest options was so that you could obtain information
> like synonyms and preferred text from a vocabulary without writing a column
> of its codes to the dictionary. It may not be working as intended or I may
> have deviated from that tactic at some point.
>
> A checked dest vocabulary will add a column to the dictionary database,
> but you can accelerate searches in ctakes by excluding them as desired code
> sources. I am probably diving too far into the weeds with that
> information.
>
> While it is possible that the ctakes dictionary creator leaves unspecified
> preferred texts unavailable for a 'good' reason, it is more likely that I
> wasn't paying enough attention during implementation. It has been 2 years
> since I looked at the dictionary creator (for the case-sensitive lookup),
> but I might revisit it on a snowy afternoon this winter. Of course, if
> anybody out there in the dev world would like to take a first crack at it
> ...
>
> Sean
> ________________________________________
> From: Peter Abramowitsch <pa...@gmail.com>
> Sent: Tuesday, December 7, 2021 12:34 PM
> To: dev@ctakes.apache.org
> Subject: Re: empty preferredText [EXTERNAL]
>
> * External Email - Caution *
>
>
> I think the issue is that preferred text in the dictionary is only
> populated by matches from the "dest" vocabularies and it uses *their*
> preferred text. If there's no match in any of them, then it should put the
> CUI's own preferred text entry in the dictionary, but it doesn't. I'm
> pretty sure It's available during the dictionary creation process, but
> probably not used.
>
> On Tue, Dec 7, 2021 at 6:22 PM Miller, Timothy <
> Timothy.Miller@childrens.harvard.edu> wrote:
>
> > OK, I thought this might be what's happening. I did check my 2021 UMLS
> > release and the cui does seem to have a preferred text but I think my
> > container is using an older release. For what it's worth the CUI is:
> > C0360554
> >
> > and a sentence that reproduces the issue in CVD with the current release
> > is:
> >
> > 'Patient had problems tolerating oral hydrocortisone.'
> >
> > I will see if I can find the older UMLS release lying around. I think the
> > right workaround for now is your suggestion of using the covered text.
> >
> > Tim
> >
> >
> > On Tue, 2021-12-07 at 17:59 +0100, Peter Abramowitsch wrote:
> >
> > * External Email - Caution *
> >
> >
> >
> > Hi Tim,
> >
> >
> > Yes, I've definitely encountered it. It happens when the concept has a
> >
> > CUI_TERM which has matched the text, but there is no corresponding entry
> in
> >
> > the SNOMED or other vocab table mapping CUI to SNOMED. The obvious
> choice
> >
> > is to use the covered text as a surrogate, but technically it could be
> PHI
> >
> > if that matters to you. The other thing is to see if there's an MSH term
> >
> > that maps using the metathesaurus. If so, including MSH in your
> dictionary
> >
> > as a src AND dest vocab will solve the problem.
> >
> >
> > Peter
> >
> >
> >
> > On Tue, Dec 7, 2021 at 5:45 PM Miller, Timothy <
> >
> > <ma...@childrens.harvard.edu>
> >
> > Timothy.Miller@childrens.harvard.edu
> >
> > > wrote:
> >
> >
> > Hello,
> >
> > I'm using the dictionary lookup (through ctakes-web-rest) and trying to
> >
> > read off the preferredText that comes back as a human-readable way to
> >
> > display the CUI. On a very small percentage, there does not seem to be
> any
> >
> > preferredText. Has anyone else encountered this? Is this a limitation of
> >
> > the underlying ontologies or a bug we can address?
> >
> > Tim
> >
> >
> >
>
Re: empty preferredText [EXTERNAL]
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
I think that you are both correct.
One or more preferred texts for a concept should be available if the user created a local copy of the umls that contains one a vocabulary that contains a preferred text. This is the first step in the process of creating a ctakes dictionary and takes place before the ctakes dictionary creator gui is started. I forget what the umls tool is called. Metamorphosys? Using default settings in metamorphosys(?) should make available a lot of preferred texts.
I think that the ctakes dictionary creator does need to have such vocabularies checked as src and dest. I think that the original intention of separate src and dest options was so that you could obtain information like synonyms and preferred text from a vocabulary without writing a column of its codes to the dictionary. It may not be working as intended or I may have deviated from that tactic at some point.
A checked dest vocabulary will add a column to the dictionary database, but you can accelerate searches in ctakes by excluding them as desired code sources. I am probably diving too far into the weeds with that information.
While it is possible that the ctakes dictionary creator leaves unspecified preferred texts unavailable for a 'good' reason, it is more likely that I wasn't paying enough attention during implementation. It has been 2 years since I looked at the dictionary creator (for the case-sensitive lookup), but I might revisit it on a snowy afternoon this winter. Of course, if anybody out there in the dev world would like to take a first crack at it ...
Sean
________________________________________
From: Peter Abramowitsch <pa...@gmail.com>
Sent: Tuesday, December 7, 2021 12:34 PM
To: dev@ctakes.apache.org
Subject: Re: empty preferredText [EXTERNAL]
* External Email - Caution *
I think the issue is that preferred text in the dictionary is only
populated by matches from the "dest" vocabularies and it uses *their*
preferred text. If there's no match in any of them, then it should put the
CUI's own preferred text entry in the dictionary, but it doesn't. I'm
pretty sure It's available during the dictionary creation process, but
probably not used.
On Tue, Dec 7, 2021 at 6:22 PM Miller, Timothy <
Timothy.Miller@childrens.harvard.edu> wrote:
> OK, I thought this might be what's happening. I did check my 2021 UMLS
> release and the cui does seem to have a preferred text but I think my
> container is using an older release. For what it's worth the CUI is:
> C0360554
>
> and a sentence that reproduces the issue in CVD with the current release
> is:
>
> 'Patient had problems tolerating oral hydrocortisone.'
>
> I will see if I can find the older UMLS release lying around. I think the
> right workaround for now is your suggestion of using the covered text.
>
> Tim
>
>
> On Tue, 2021-12-07 at 17:59 +0100, Peter Abramowitsch wrote:
>
> * External Email - Caution *
>
>
>
> Hi Tim,
>
>
> Yes, I've definitely encountered it. It happens when the concept has a
>
> CUI_TERM which has matched the text, but there is no corresponding entry in
>
> the SNOMED or other vocab table mapping CUI to SNOMED. The obvious choice
>
> is to use the covered text as a surrogate, but technically it could be PHI
>
> if that matters to you. The other thing is to see if there's an MSH term
>
> that maps using the metathesaurus. If so, including MSH in your dictionary
>
> as a src AND dest vocab will solve the problem.
>
>
> Peter
>
>
>
> On Tue, Dec 7, 2021 at 5:45 PM Miller, Timothy <
>
> <ma...@childrens.harvard.edu>
>
> Timothy.Miller@childrens.harvard.edu
>
> > wrote:
>
>
> Hello,
>
> I'm using the dictionary lookup (through ctakes-web-rest) and trying to
>
> read off the preferredText that comes back as a human-readable way to
>
> display the CUI. On a very small percentage, there does not seem to be any
>
> preferredText. Has anyone else encountered this? Is this a limitation of
>
> the underlying ontologies or a bug we can address?
>
> Tim
>
>
>
Re: empty preferredText [EXTERNAL]
Posted by Peter Abramowitsch <pa...@gmail.com>.
I think the issue is that preferred text in the dictionary is only
populated by matches from the "dest" vocabularies and it uses *their*
preferred text. If there's no match in any of them, then it should put the
CUI's own preferred text entry in the dictionary, but it doesn't. I'm
pretty sure It's available during the dictionary creation process, but
probably not used.
On Tue, Dec 7, 2021 at 6:22 PM Miller, Timothy <
Timothy.Miller@childrens.harvard.edu> wrote:
> OK, I thought this might be what's happening. I did check my 2021 UMLS
> release and the cui does seem to have a preferred text but I think my
> container is using an older release. For what it's worth the CUI is:
> C0360554
>
> and a sentence that reproduces the issue in CVD with the current release
> is:
>
> 'Patient had problems tolerating oral hydrocortisone.'
>
> I will see if I can find the older UMLS release lying around. I think the
> right workaround for now is your suggestion of using the covered text.
>
> Tim
>
>
> On Tue, 2021-12-07 at 17:59 +0100, Peter Abramowitsch wrote:
>
> * External Email - Caution *
>
>
>
> Hi Tim,
>
>
> Yes, I've definitely encountered it. It happens when the concept has a
>
> CUI_TERM which has matched the text, but there is no corresponding entry in
>
> the SNOMED or other vocab table mapping CUI to SNOMED. The obvious choice
>
> is to use the covered text as a surrogate, but technically it could be PHI
>
> if that matters to you. The other thing is to see if there's an MSH term
>
> that maps using the metathesaurus. If so, including MSH in your dictionary
>
> as a src AND dest vocab will solve the problem.
>
>
> Peter
>
>
>
> On Tue, Dec 7, 2021 at 5:45 PM Miller, Timothy <
>
> <ma...@childrens.harvard.edu>
>
> Timothy.Miller@childrens.harvard.edu
>
> > wrote:
>
>
> Hello,
>
> I'm using the dictionary lookup (through ctakes-web-rest) and trying to
>
> read off the preferredText that comes back as a human-readable way to
>
> display the CUI. On a very small percentage, there does not seem to be any
>
> preferredText. Has anyone else encountered this? Is this a limitation of
>
> the underlying ontologies or a bug we can address?
>
> Tim
>
>
>