You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by Shady Hussein <sh...@gmail.com> on 2013/02/04 11:46:51 UTC
Combining Drug and Concept annotations
Dear All,
I discovered that cTAKES doesn't recognize all the medical entities as
concepts. There is a difference between using the normal UMLS dictionary in
"/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
"/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.xml". The
later can detect all the drugs, while the first can't.
My question now is how to combine between both of those dictionaries. So i
can detect all the drugs and concepts mentioned in the text. I would be
grateful if somebody can help me :)
--
Thanks and best Regards,
Shady AbdelAziz
Re: Combining Drug and Concept annotations
Posted by Shady Hussein <sh...@gmail.com>.
The last change improved the results a lot. Currently i am using the
descriptor you sent me plus the last change and i think the results are
very satisfiable.
Iam still investigating other stuff regarding the detection and filtering
of the cTAKES. cause sometimes entities are detected and some other times
they are not. I need to know what is happening and what is the pattern. I
will keep you updated with the results
Thanks all for your consideration and attention
On Fri, Feb 8, 2013 at 3:42 AM, Masanz, James J. <Ma...@mayo.edu>wrote:
> In LookupDesc_Db.xml, trying changing this line
> <lookupConsumer
> className="edu.mayo.bmi.uima.lookup.ae.OrangeBookFilterConsumerImpl">
> To this:
> <lookupConsumer
> className="edu.mayo.bmi.uima.lookup.ae.NamedEntityLookupConsumerImpl">
>
> -- James
>
> > -----Original Message-----
> > From: ctakes-dev-return-1172-Masanz.James=mayo.edu@incubator.apache.org
> > [mailto:ctakes-dev-return-1172-Masanz.James=
> mayo.edu@incubator.apache.org]
> > On Behalf Of Masanz, James J.
> > Sent: Thursday, February 07, 2013 1:33 PM
> > To: 'ctakes-dev@incubator.apache.org'
> > Subject: RE: Combining Drug and Concept annotations
> >
> > The clinical documents pipeline
> > (cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml) filters the
> > drug mentions it finds, using the Orange Book.
> > The drug NER pipeline does not do that filtering.
> > I suspect that accounts for the differences you are seeing.
> > The pipeline logs a message when it filters out a drug due to not finding
> > it in the version of the Orange Book that is included within cTAKES.
> >
> > If you need pointers on how to remove that filtering, I can take a look
> at
> > that later today.
> >
> > -- James
> >
> > > -----Original Message-----
> > > From:
> > > ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.org
> > > [mailto:ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.
> > > org]
> > > On Behalf Of Shady Hussein
> > > Sent: Wednesday, February 06, 2013 4:21 AM
> > > To: ctakes-dev@incubator.apache.org
> > > Subject: Re: Combining Drug and Concept annotations
> > >
> > > I tried it but it doesn't give me the required results. Now it
> > > annotates the drugs as "Concept" and "MedicationEventMention" but
> > > there are still some drugs which are not annotated while it is
> > > annotated as "MedicationEventMention" if i used
> > > "DrugAggregatePlaintextUMLSProcessor.xml". You can try this scenario
> > > with this text
> > >
> > > DISCHARGE MEDICATIONS :
> > > 1. Tegretol 200 mg p.o. t.i.d.
> > > 2. Klonopin 1 mg p.o. b.i.d.
> > > 3. Colace 100 mg p.o. b.i.d.
> > > 4. Nicoderm 21 mg q d. topical .
> > > 5. Zofran 24 mg p.o. q d. x 3 days .
> > >
> > > Here "Colace" and "Nicoderm" are not annotated but are annotated as
> > > "MedicationEventMention" when you use
> > > "DrugAggregatePlaintextUMLSProcessor.xml".
> > >
> > > Thanks for your help and looking forward to your reply
> > >
> > > On Wed, Feb 6, 2013 at 3:14 AM, Masanz, James J.
> > > <Ma...@mayo.edu>wrote:
> > >
> > > >
> > > > I'm attaching something you can try.
> > > >
> > > > I'm assuming you are using cTAKES 2.5.
> > > > 1) put the attached file in cTAKESdesc\cdpdesc\analysis_engine
> > > > 2) update
> > > > cTAKESdesc\lookupdesc\analysis_engine\DictionaryLookupAnnotatorUMLS.
> > > > xm
> > > > l
> > > > with your UMLS username and password
> > > > 3) update the other copy of DictionaryLookupAnnotatorUMLS.xml in
> > > > cTAKESdesc\drugnerdesc\analysis_engine also with your UMLS username
> > > > and password
> > > >
> > > > -- James Masanz
> > > > ________________________________________
> > > > From:
> > > > ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org[ct
> > > > ak es-dev-return-1136-Masanz.James= mayo.edu@incubator.apache.org]
> > > > on behalf of shady hussein [ shady.aziz@gmail.com]
> > > > Sent: Monday, February 04, 2013 1:52 PM
> > > > To: ctakes-dev@incubator.apache.org
> > > > Subject: Re: Combining Drug and Concept annotations
> > > >
> > > > Hi Pei,
> > > > Thanks for your reply, Yes i meant that
> > > > DrugAggregatePlaintextUMLSProcessor return more concepts, or the
> > > > opposite AggregatePlaintextUMLSProcessor returns the usual concepts
> > > > + the MedicationEventMentions. I don't think it is hard to
> > > > implement, as i think the dictionary lookup code won't change.
> > > >
> > > > I tried to merge the drug lookup in the lookupDB and add the drug
> > > > annotator in the normal pipe line, but of course things are not that
> > > > simple
> > > > :) I don't fully understand how the dictionary look up works,
> > > > otherwise i could do it. Maybe if you have sometime, you can guide
> > > > me a little and i can go from there.
> > > >
> > > > Thanks,
> > > > Shady
> > > >
> > > > On Feb 4, 2013, at 6:58 PM, "Chen, Pei"
> > > > <Pe...@childrens.harvard.edu>
> > > > wrote:
> > > >
> > > > > Hi Shady,
> > > > > Just wanted to confirm:
> > > > > Did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > > > identifies
> > > > the same drugs, but just with more attributes (i.e. dosage,
> > > > frequency, etc.)?
> > > > > Or did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > > > actually
> > > > returned more UMLSConcepts (MedicationEventMentions) the regular
> > > > AggregatePlaintextUMLSProcessor?
> > > > >
> > > > > For the former, there is an outstanding Jira item to combine the 2
> > > > (reusing the existing lookup entries- rather than a second lookup):
> > > > https://issues.apache.org/jira/browse/CTAKES-20
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Shady Hussein [mailto:shady.aziz@gmail.com]
> > > > >> Sent: Monday, February 04, 2013 5:47 AM
> > > > >> To: ctakes-dev@incubator.apache.org
> > > > >> Subject: Combining Drug and Concept annotations
> > > > >>
> > > > >> Dear All,
> > > > >> I discovered that cTAKES doesn't recognize all the medical
> > > > >> entities as concepts. There is a difference between using the
> > > > >> normal UMLS
> > > > dictionary in
> > > > >> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml"
> > > > >> and
> > > > >> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor
> > > > >> .x ml ". The later can detect all the drugs, while the first
> > > > >> can't.
> > > > >>
> > > > >> My question now is how to combine between both of those
> > dictionaries.
> > > > So i
> > > > >> can detect all the drugs and concepts mentioned in the text. I
> > > > >> would be grateful if somebody can help me :)
> > > > >>
> > > > >> --
> > > > >> Thanks and best Regards,
> > > > >>
> > > > >> Shady AbdelAziz
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks and best Regards,
> > >
> > > Shady AbdelAziz
>
--
Thanks and best Regards,
Shady AbdelAziz
RE: Combining Drug and Concept annotations
Posted by "Masanz, James J." <Ma...@mayo.edu>.
In LookupDesc_Db.xml, trying changing this line
<lookupConsumer className="edu.mayo.bmi.uima.lookup.ae.OrangeBookFilterConsumerImpl">
To this:
<lookupConsumer className="edu.mayo.bmi.uima.lookup.ae.NamedEntityLookupConsumerImpl">
-- James
> -----Original Message-----
> From: ctakes-dev-return-1172-Masanz.James=mayo.edu@incubator.apache.org
> [mailto:ctakes-dev-return-1172-Masanz.James=mayo.edu@incubator.apache.org]
> On Behalf Of Masanz, James J.
> Sent: Thursday, February 07, 2013 1:33 PM
> To: 'ctakes-dev@incubator.apache.org'
> Subject: RE: Combining Drug and Concept annotations
>
> The clinical documents pipeline
> (cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml) filters the
> drug mentions it finds, using the Orange Book.
> The drug NER pipeline does not do that filtering.
> I suspect that accounts for the differences you are seeing.
> The pipeline logs a message when it filters out a drug due to not finding
> it in the version of the Orange Book that is included within cTAKES.
>
> If you need pointers on how to remove that filtering, I can take a look at
> that later today.
>
> -- James
>
> > -----Original Message-----
> > From:
> > ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.org
> > [mailto:ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.
> > org]
> > On Behalf Of Shady Hussein
> > Sent: Wednesday, February 06, 2013 4:21 AM
> > To: ctakes-dev@incubator.apache.org
> > Subject: Re: Combining Drug and Concept annotations
> >
> > I tried it but it doesn't give me the required results. Now it
> > annotates the drugs as "Concept" and "MedicationEventMention" but
> > there are still some drugs which are not annotated while it is
> > annotated as "MedicationEventMention" if i used
> > "DrugAggregatePlaintextUMLSProcessor.xml". You can try this scenario
> > with this text
> >
> > DISCHARGE MEDICATIONS :
> > 1. Tegretol 200 mg p.o. t.i.d.
> > 2. Klonopin 1 mg p.o. b.i.d.
> > 3. Colace 100 mg p.o. b.i.d.
> > 4. Nicoderm 21 mg q d. topical .
> > 5. Zofran 24 mg p.o. q d. x 3 days .
> >
> > Here "Colace" and "Nicoderm" are not annotated but are annotated as
> > "MedicationEventMention" when you use
> > "DrugAggregatePlaintextUMLSProcessor.xml".
> >
> > Thanks for your help and looking forward to your reply
> >
> > On Wed, Feb 6, 2013 at 3:14 AM, Masanz, James J.
> > <Ma...@mayo.edu>wrote:
> >
> > >
> > > I'm attaching something you can try.
> > >
> > > I'm assuming you are using cTAKES 2.5.
> > > 1) put the attached file in cTAKESdesc\cdpdesc\analysis_engine
> > > 2) update
> > > cTAKESdesc\lookupdesc\analysis_engine\DictionaryLookupAnnotatorUMLS.
> > > xm
> > > l
> > > with your UMLS username and password
> > > 3) update the other copy of DictionaryLookupAnnotatorUMLS.xml in
> > > cTAKESdesc\drugnerdesc\analysis_engine also with your UMLS username
> > > and password
> > >
> > > -- James Masanz
> > > ________________________________________
> > > From:
> > > ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org[ct
> > > ak es-dev-return-1136-Masanz.James= mayo.edu@incubator.apache.org]
> > > on behalf of shady hussein [ shady.aziz@gmail.com]
> > > Sent: Monday, February 04, 2013 1:52 PM
> > > To: ctakes-dev@incubator.apache.org
> > > Subject: Re: Combining Drug and Concept annotations
> > >
> > > Hi Pei,
> > > Thanks for your reply, Yes i meant that
> > > DrugAggregatePlaintextUMLSProcessor return more concepts, or the
> > > opposite AggregatePlaintextUMLSProcessor returns the usual concepts
> > > + the MedicationEventMentions. I don't think it is hard to
> > > implement, as i think the dictionary lookup code won't change.
> > >
> > > I tried to merge the drug lookup in the lookupDB and add the drug
> > > annotator in the normal pipe line, but of course things are not that
> > > simple
> > > :) I don't fully understand how the dictionary look up works,
> > > otherwise i could do it. Maybe if you have sometime, you can guide
> > > me a little and i can go from there.
> > >
> > > Thanks,
> > > Shady
> > >
> > > On Feb 4, 2013, at 6:58 PM, "Chen, Pei"
> > > <Pe...@childrens.harvard.edu>
> > > wrote:
> > >
> > > > Hi Shady,
> > > > Just wanted to confirm:
> > > > Did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > > identifies
> > > the same drugs, but just with more attributes (i.e. dosage,
> > > frequency, etc.)?
> > > > Or did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > > actually
> > > returned more UMLSConcepts (MedicationEventMentions) the regular
> > > AggregatePlaintextUMLSProcessor?
> > > >
> > > > For the former, there is an outstanding Jira item to combine the 2
> > > (reusing the existing lookup entries- rather than a second lookup):
> > > https://issues.apache.org/jira/browse/CTAKES-20
> > > >
> > > >> -----Original Message-----
> > > >> From: Shady Hussein [mailto:shady.aziz@gmail.com]
> > > >> Sent: Monday, February 04, 2013 5:47 AM
> > > >> To: ctakes-dev@incubator.apache.org
> > > >> Subject: Combining Drug and Concept annotations
> > > >>
> > > >> Dear All,
> > > >> I discovered that cTAKES doesn't recognize all the medical
> > > >> entities as concepts. There is a difference between using the
> > > >> normal UMLS
> > > dictionary in
> > > >> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml"
> > > >> and
> > > >> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor
> > > >> .x ml ". The later can detect all the drugs, while the first
> > > >> can't.
> > > >>
> > > >> My question now is how to combine between both of those
> dictionaries.
> > > So i
> > > >> can detect all the drugs and concepts mentioned in the text. I
> > > >> would be grateful if somebody can help me :)
> > > >>
> > > >> --
> > > >> Thanks and best Regards,
> > > >>
> > > >> Shady AbdelAziz
> > >
> >
> >
> >
> > --
> > Thanks and best Regards,
> >
> > Shady AbdelAziz
RE: Combining Drug and Concept annotations
Posted by "Masanz, James J." <Ma...@mayo.edu>.
The clinical documents pipeline (cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml) filters the drug mentions it finds, using the Orange Book.
The drug NER pipeline does not do that filtering.
I suspect that accounts for the differences you are seeing.
The pipeline logs a message when it filters out a drug due to not finding it in the version of the Orange Book that is included within cTAKES.
If you need pointers on how to remove that filtering, I can take a look at that later today.
-- James
> -----Original Message-----
> From: ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.org
> [mailto:ctakes-dev-return-1156-Masanz.James=mayo.edu@incubator.apache.org]
> On Behalf Of Shady Hussein
> Sent: Wednesday, February 06, 2013 4:21 AM
> To: ctakes-dev@incubator.apache.org
> Subject: Re: Combining Drug and Concept annotations
>
> I tried it but it doesn't give me the required results. Now it annotates
> the drugs as "Concept" and "MedicationEventMention" but there are still
> some drugs which are not annotated while it is annotated as
> "MedicationEventMention" if i used
> "DrugAggregatePlaintextUMLSProcessor.xml". You can try this scenario with
> this text
>
> DISCHARGE MEDICATIONS :
> 1. Tegretol 200 mg p.o. t.i.d.
> 2. Klonopin 1 mg p.o. b.i.d.
> 3. Colace 100 mg p.o. b.i.d.
> 4. Nicoderm 21 mg q d. topical .
> 5. Zofran 24 mg p.o. q d. x 3 days .
>
> Here "Colace" and "Nicoderm" are not annotated but are annotated as
> "MedicationEventMention" when you use
> "DrugAggregatePlaintextUMLSProcessor.xml".
>
> Thanks for your help and looking forward to your reply
>
> On Wed, Feb 6, 2013 at 3:14 AM, Masanz, James J.
> <Ma...@mayo.edu>wrote:
>
> >
> > I'm attaching something you can try.
> >
> > I'm assuming you are using cTAKES 2.5.
> > 1) put the attached file in cTAKESdesc\cdpdesc\analysis_engine
> > 2) update
> > cTAKESdesc\lookupdesc\analysis_engine\DictionaryLookupAnnotatorUMLS.xm
> > l
> > with your UMLS username and password
> > 3) update the other copy of DictionaryLookupAnnotatorUMLS.xml in
> > cTAKESdesc\drugnerdesc\analysis_engine also with your UMLS username
> > and password
> >
> > -- James Masanz
> > ________________________________________
> > From:
> > ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org[ctak
> > es-dev-return-1136-Masanz.James= mayo.edu@incubator.apache.org] on
> > behalf of shady hussein [ shady.aziz@gmail.com]
> > Sent: Monday, February 04, 2013 1:52 PM
> > To: ctakes-dev@incubator.apache.org
> > Subject: Re: Combining Drug and Concept annotations
> >
> > Hi Pei,
> > Thanks for your reply, Yes i meant that
> > DrugAggregatePlaintextUMLSProcessor return more concepts, or the
> > opposite AggregatePlaintextUMLSProcessor returns the usual concepts +
> > the MedicationEventMentions. I don't think it is hard to implement, as
> > i think the dictionary lookup code won't change.
> >
> > I tried to merge the drug lookup in the lookupDB and add the drug
> > annotator in the normal pipe line, but of course things are not that
> > simple
> > :) I don't fully understand how the dictionary look up works,
> > otherwise i could do it. Maybe if you have sometime, you can guide me
> > a little and i can go from there.
> >
> > Thanks,
> > Shady
> >
> > On Feb 4, 2013, at 6:58 PM, "Chen, Pei"
> > <Pe...@childrens.harvard.edu>
> > wrote:
> >
> > > Hi Shady,
> > > Just wanted to confirm:
> > > Did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > identifies
> > the same drugs, but just with more attributes (i.e. dosage, frequency,
> > etc.)?
> > > Or did you mean that the DrugAggregatePlaintextUMLSProcessor
> > > actually
> > returned more UMLSConcepts (MedicationEventMentions) the regular
> > AggregatePlaintextUMLSProcessor?
> > >
> > > For the former, there is an outstanding Jira item to combine the 2
> > (reusing the existing lookup entries- rather than a second lookup):
> > https://issues.apache.org/jira/browse/CTAKES-20
> > >
> > >> -----Original Message-----
> > >> From: Shady Hussein [mailto:shady.aziz@gmail.com]
> > >> Sent: Monday, February 04, 2013 5:47 AM
> > >> To: ctakes-dev@incubator.apache.org
> > >> Subject: Combining Drug and Concept annotations
> > >>
> > >> Dear All,
> > >> I discovered that cTAKES doesn't recognize all the medical
> > >> entities as concepts. There is a difference between using the
> > >> normal UMLS
> > dictionary in
> > >> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
> > >> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.x
> > >> ml ". The later can detect all the drugs, while the first can't.
> > >>
> > >> My question now is how to combine between both of those dictionaries.
> > So i
> > >> can detect all the drugs and concepts mentioned in the text. I
> > >> would be grateful if somebody can help me :)
> > >>
> > >> --
> > >> Thanks and best Regards,
> > >>
> > >> Shady AbdelAziz
> >
>
>
>
> --
> Thanks and best Regards,
>
> Shady AbdelAziz
Re: Combining Drug and Concept annotations
Posted by Shady Hussein <sh...@gmail.com>.
I tried it but it doesn't give me the required results. Now it annotates
the drugs as "Concept" and "MedicationEventMention" but there are still
some drugs which are not annotated while it is annotated as
"MedicationEventMention" if i used
"DrugAggregatePlaintextUMLSProcessor.xml". You can try this scenario with
this text
DISCHARGE MEDICATIONS :
1. Tegretol 200 mg p.o. t.i.d.
2. Klonopin 1 mg p.o. b.i.d.
3. Colace 100 mg p.o. b.i.d.
4. Nicoderm 21 mg q d. topical .
5. Zofran 24 mg p.o. q d. x 3 days .
Here "Colace" and "Nicoderm" are not annotated but are annotated as
"MedicationEventMention" when you use
"DrugAggregatePlaintextUMLSProcessor.xml".
Thanks for your help and looking forward to your reply
On Wed, Feb 6, 2013 at 3:14 AM, Masanz, James J. <Ma...@mayo.edu>wrote:
>
> I'm attaching something you can try.
>
> I'm assuming you are using cTAKES 2.5.
> 1) put the attached file in cTAKESdesc\cdpdesc\analysis_engine
> 2) update
> cTAKESdesc\lookupdesc\analysis_engine\DictionaryLookupAnnotatorUMLS.xml
> with your UMLS username and password
> 3) update the other copy of DictionaryLookupAnnotatorUMLS.xml in
> cTAKESdesc\drugnerdesc\analysis_engine also with your UMLS username and
> password
>
> -- James Masanz
> ________________________________________
> From: ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org[ctakes-dev-return-1136-Masanz.James=
> mayo.edu@incubator.apache.org] on behalf of shady hussein [
> shady.aziz@gmail.com]
> Sent: Monday, February 04, 2013 1:52 PM
> To: ctakes-dev@incubator.apache.org
> Subject: Re: Combining Drug and Concept annotations
>
> Hi Pei,
> Thanks for your reply, Yes i meant that
> DrugAggregatePlaintextUMLSProcessor return more concepts, or the opposite
> AggregatePlaintextUMLSProcessor returns the usual concepts + the
> MedicationEventMentions. I don't think it is hard to implement, as i think
> the dictionary lookup code won't change.
>
> I tried to merge the drug lookup in the lookupDB and add the drug
> annotator in the normal pipe line, but of course things are not that simple
> :) I don't fully understand how the dictionary look up works, otherwise i
> could do it. Maybe if you have sometime, you can guide me a little and i
> can go from there.
>
> Thanks,
> Shady
>
> On Feb 4, 2013, at 6:58 PM, "Chen, Pei" <Pe...@childrens.harvard.edu>
> wrote:
>
> > Hi Shady,
> > Just wanted to confirm:
> > Did you mean that the DrugAggregatePlaintextUMLSProcessor identifies
> the same drugs, but just with more attributes (i.e. dosage, frequency,
> etc.)?
> > Or did you mean that the DrugAggregatePlaintextUMLSProcessor actually
> returned more UMLSConcepts (MedicationEventMentions) the regular
> AggregatePlaintextUMLSProcessor?
> >
> > For the former, there is an outstanding Jira item to combine the 2
> (reusing the existing lookup entries- rather than a second lookup):
> https://issues.apache.org/jira/browse/CTAKES-20
> >
> >> -----Original Message-----
> >> From: Shady Hussein [mailto:shady.aziz@gmail.com]
> >> Sent: Monday, February 04, 2013 5:47 AM
> >> To: ctakes-dev@incubator.apache.org
> >> Subject: Combining Drug and Concept annotations
> >>
> >> Dear All,
> >> I discovered that cTAKES doesn't recognize all the medical entities as
> >> concepts. There is a difference between using the normal UMLS
> dictionary in
> >> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
> >> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.xml
> >> ". The later can detect all the drugs, while the first can't.
> >>
> >> My question now is how to combine between both of those dictionaries.
> So i
> >> can detect all the drugs and concepts mentioned in the text. I would be
> >> grateful if somebody can help me :)
> >>
> >> --
> >> Thanks and best Regards,
> >>
> >> Shady AbdelAziz
>
--
Thanks and best Regards,
Shady AbdelAziz
RE: Combining Drug and Concept annotations
Posted by "Masanz, James J." <Ma...@mayo.edu>.
I'm attaching something you can try.
I'm assuming you are using cTAKES 2.5.
1) put the attached file in cTAKESdesc\cdpdesc\analysis_engine
2) update cTAKESdesc\lookupdesc\analysis_engine\DictionaryLookupAnnotatorUMLS.xml with your UMLS username and password
3) update the other copy of DictionaryLookupAnnotatorUMLS.xml in cTAKESdesc\drugnerdesc\analysis_engine also with your UMLS username and password
-- James Masanz
________________________________________
From: ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org [ctakes-dev-return-1136-Masanz.James=mayo.edu@incubator.apache.org] on behalf of shady hussein [shady.aziz@gmail.com]
Sent: Monday, February 04, 2013 1:52 PM
To: ctakes-dev@incubator.apache.org
Subject: Re: Combining Drug and Concept annotations
Hi Pei,
Thanks for your reply, Yes i meant that DrugAggregatePlaintextUMLSProcessor return more concepts, or the opposite AggregatePlaintextUMLSProcessor returns the usual concepts + the MedicationEventMentions. I don't think it is hard to implement, as i think the dictionary lookup code won't change.
I tried to merge the drug lookup in the lookupDB and add the drug annotator in the normal pipe line, but of course things are not that simple :) I don't fully understand how the dictionary look up works, otherwise i could do it. Maybe if you have sometime, you can guide me a little and i can go from there.
Thanks,
Shady
On Feb 4, 2013, at 6:58 PM, "Chen, Pei" <Pe...@childrens.harvard.edu> wrote:
> Hi Shady,
> Just wanted to confirm:
> Did you mean that the DrugAggregatePlaintextUMLSProcessor identifies the same drugs, but just with more attributes (i.e. dosage, frequency, etc.)?
> Or did you mean that the DrugAggregatePlaintextUMLSProcessor actually returned more UMLSConcepts (MedicationEventMentions) the regular AggregatePlaintextUMLSProcessor?
>
> For the former, there is an outstanding Jira item to combine the 2 (reusing the existing lookup entries- rather than a second lookup): https://issues.apache.org/jira/browse/CTAKES-20
>
>> -----Original Message-----
>> From: Shady Hussein [mailto:shady.aziz@gmail.com]
>> Sent: Monday, February 04, 2013 5:47 AM
>> To: ctakes-dev@incubator.apache.org
>> Subject: Combining Drug and Concept annotations
>>
>> Dear All,
>> I discovered that cTAKES doesn't recognize all the medical entities as
>> concepts. There is a difference between using the normal UMLS dictionary in
>> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
>> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.xml
>> ". The later can detect all the drugs, while the first can't.
>>
>> My question now is how to combine between both of those dictionaries. So i
>> can detect all the drugs and concepts mentioned in the text. I would be
>> grateful if somebody can help me :)
>>
>> --
>> Thanks and best Regards,
>>
>> Shady AbdelAziz
Re: Combining Drug and Concept annotations
Posted by shady hussein <sh...@gmail.com>.
Hi Pei,
Thanks for your reply, Yes i meant that DrugAggregatePlaintextUMLSProcessor return more concepts, or the opposite AggregatePlaintextUMLSProcessor returns the usual concepts + the MedicationEventMentions. I don't think it is hard to implement, as i think the dictionary lookup code won't change.
I tried to merge the drug lookup in the lookupDB and add the drug annotator in the normal pipe line, but of course things are not that simple :) I don't fully understand how the dictionary look up works, otherwise i could do it. Maybe if you have sometime, you can guide me a little and i can go from there.
Thanks,
Shady
On Feb 4, 2013, at 6:58 PM, "Chen, Pei" <Pe...@childrens.harvard.edu> wrote:
> Hi Shady,
> Just wanted to confirm:
> Did you mean that the DrugAggregatePlaintextUMLSProcessor identifies the same drugs, but just with more attributes (i.e. dosage, frequency, etc.)?
> Or did you mean that the DrugAggregatePlaintextUMLSProcessor actually returned more UMLSConcepts (MedicationEventMentions) the regular AggregatePlaintextUMLSProcessor?
>
> For the former, there is an outstanding Jira item to combine the 2 (reusing the existing lookup entries- rather than a second lookup): https://issues.apache.org/jira/browse/CTAKES-20
>
>> -----Original Message-----
>> From: Shady Hussein [mailto:shady.aziz@gmail.com]
>> Sent: Monday, February 04, 2013 5:47 AM
>> To: ctakes-dev@incubator.apache.org
>> Subject: Combining Drug and Concept annotations
>>
>> Dear All,
>> I discovered that cTAKES doesn't recognize all the medical entities as
>> concepts. There is a difference between using the normal UMLS dictionary in
>> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
>> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.xml
>> ". The later can detect all the drugs, while the first can't.
>>
>> My question now is how to combine between both of those dictionaries. So i
>> can detect all the drugs and concepts mentioned in the text. I would be
>> grateful if somebody can help me :)
>>
>> --
>> Thanks and best Regards,
>>
>> Shady AbdelAziz
RE: Combining Drug and Concept annotations
Posted by "Chen, Pei" <Pe...@childrens.harvard.edu>.
Hi Shady,
Just wanted to confirm:
Did you mean that the DrugAggregatePlaintextUMLSProcessor identifies the same drugs, but just with more attributes (i.e. dosage, frequency, etc.)?
Or did you mean that the DrugAggregatePlaintextUMLSProcessor actually returned more UMLSConcepts (MedicationEventMentions) the regular AggregatePlaintextUMLSProcessor?
For the former, there is an outstanding Jira item to combine the 2 (reusing the existing lookup entries- rather than a second lookup): https://issues.apache.org/jira/browse/CTAKES-20
> -----Original Message-----
> From: Shady Hussein [mailto:shady.aziz@gmail.com]
> Sent: Monday, February 04, 2013 5:47 AM
> To: ctakes-dev@incubator.apache.org
> Subject: Combining Drug and Concept annotations
>
> Dear All,
> I discovered that cTAKES doesn't recognize all the medical entities as
> concepts. There is a difference between using the normal UMLS dictionary in
> "/cdpdesc/analysis_engine/AggregatePlaintextUMLSProcessor.xml" and
> "/drugnerdesc/analysis_engine/DrugAggregatePlaintextUMLSProcessor.xml
> ". The later can detect all the drugs, while the first can't.
>
> My question now is how to combine between both of those dictionaries. So i
> can detect all the drugs and concepts mentioned in the text. I would be
> grateful if somebody can help me :)
>
> --
> Thanks and best Regards,
>
> Shady AbdelAziz