You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ctakes.apache.org by Ory Henn <or...@trialjectory.com> on 2018/08/20 08:22:25 UTC
How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am
trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I
specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
RE: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Gandhi Rajan Natarajan <Ga...@arisglobal.com>.
Glad to hear that you made it work Ory.
Regards,
Gandhi
From: Ory Henn <or...@trialjectory.com>
Sent: Monday, September 3, 2018 9:44 PM
To: user@ctakes.apache.org
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
It works!
Thanks Gandhi and everyone else who helped, let me once again say the experience as a newbie in this group is amazing.
FWIW, and future readers, I found the relevant information in the UIMA references: https://uima.apache.org/downloads/releaseDocs/2.2.2-incubating/docs/html/references/references.html#ugr.ref.xml.cpe_descriptor, linked from the tutorials mentioned above.
Ory
On Tue, Aug 28, 2018 at 9:11 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
Piper files are equivalent of the XML descriptor files which helps in adding required analysis engines.
You can manipulate analysis engines for CPE using CPE configurator.
Check this link for more info - http://uima.apache.org/downloads/releaseDocs/2.3.0-incubating/docs/html/tutorials_and_users_guides/tutorials_and_users_guides.html#ugr.tug.cpe.configurator_and_viewer
Regards,
Gandhi
From: Ory Henn <or...@trialjectory.com>>
Sent: Sunday, August 26, 2018 5:25 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks, Gandhi.
I took a look at the module, but am still a bit confused, as the connection between the CPE tool and the piper files is not clear to me.
I've succeeded in using my own dictionary with the Default Clinical Pipeline, both through -l argument and through adding a LookupXml parameter in the DefaultFastPipeline.piper script (as in ctakes-rest-service above).
But when I run the CPE tool, it still references the default dictionary.
My question is - what is the link betwenn CVD/CPE tools and the piper files? Specifically, can I configure the CPE tool to use my piper file, or configure it to use a different lookup?
Many thanks,
Ory
On Thu, Aug 23, 2018 at 8:04 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I suggest you to have a look at this ctakes rest module - https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes-web-rest
The custom dictionary XML needs to be placed in ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as follows:
add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/dictionary/lookup/fast/customDictionary.xml
From: Ory Henn <or...@trialjectory.com>>
Sent: Wednesday, August 22, 2018 8:42 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks, Timothy.
I fixed the permissions issue and was able to progress further.
I am now stuck at the very last step (it would seem), the Dictionary Creator GUI page says this:
---
Once a new dictionary has been built, point to it in one of 2 ways:
Set the fast dictionary parameter LookupXml to org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
---
Where is this LookupXml parameter to be found/added?
On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <Ti...@childrens.harvard.edu>> wrote:
Looks like maybe you don't have permissions to write to that directory (/usr/local/). Can you write it somewhere else?
________________________________
From: Ory Henn <or...@trialjectory.com>>
Sent: Wednesday, August 22, 2018 9:15 AM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks a lot Ghandi, Michael, for quick and helpful replies.
The video especially was very useful, but I'm encountering an error at the last part (building the actual dictionary).
It looks like it has a problem with an HsqlDB table (log output below).
Can anyone help with this? Do I need to install HsqlDB, or does it come as part of UMLS/Metamorphosys?
Thanks,
Ory
22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts: 450236 Texts: 1380174
22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD ...
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts: 1162474
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or object not found: NCI_NCI in statement [insert into nci_nci_hl7 (CUI,NCI_NCI-HL7) values (?,?)]
22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not be built in /usr/local/apache-ctakes-4.0.0
On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>> wrote:
Ory,
In response to Gandhi's comments, the video below outlines custom dictionary creation in detail:
https://www.youtube.com/watch?v=4aOnafv-NQs<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
Best,
Mike
On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you want to lookup from other dictionaries like MEDDRA etc. , you may have to create your custom dictionary using cTAKES dictionary generator GUI. That’s what I did to include MEDDRA dictionary terms.
-----Original Message-----
From: Ory Henn <or...@trialjectory.com>>
Sent: Monday, August 20, 2018 1:52 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Cc: Guy Gildor <gu...@trialjectory.com>>
Subject: How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=> <https://trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
--
[MetiStream Logo - 500]
Mike Trepanier| Big Data Engineer | MetiStream, Inc. | mike@metistream.com<ma...@metistream.com> | 845 - 270 - 3129 (m) | www.metistream.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Ory Henn <or...@trialjectory.com>.
It works!
Thanks Gandhi and everyone else who helped, let me once again say the
experience as a newbie in this group is amazing.
FWIW, and future readers, I found the relevant information in the UIMA
references:
https://uima.apache.org/downloads/releaseDocs/2.2.2-incubating/docs/html/references/references.html#ugr.ref.xml.cpe_descriptor,
linked from the tutorials mentioned above.
Ory
On Tue, Aug 28, 2018 at 9:11 AM, Gandhi Rajan Natarajan <
Gandhi.Natarajan@arisglobal.com> wrote:
> Hi Ory,
>
>
>
> Piper files are equivalent of the XML descriptor files which helps in
> adding required analysis engines.
>
>
>
> You can manipulate analysis engines for CPE using CPE configurator.
>
>
>
> Check this link for more info - http://uima.apache.org/
> downloads/releaseDocs/2.3.0-incubating/docs/html/
> tutorials_and_users_guides/tutorials_and_users_guides.
> html#ugr.tug.cpe.configurator_and_viewer
>
>
>
> Regards,
>
> Gandhi
>
>
>
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Sunday, August 26, 2018 5:25 PM
>
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
>
>
> Thanks, Gandhi.
>
> I took a look at the module, but am still a bit confused, as the
> connection between the CPE tool and the piper files is not clear to me.
>
> I've succeeded in using my own dictionary with the Default Clinical
> Pipeline, both through -l argument and through adding a LookupXml parameter
> in the DefaultFastPipeline.piper script (as in ctakes-rest-service above).
>
> But when I run the CPE tool, it still references the default dictionary.
>
> My question is - what is the link betwenn CVD/CPE tools and the piper
> files? Specifically, can I configure the CPE tool to use my piper file, or
> configure it to use a different lookup?
>
> Many thanks,
>
> Ory
>
>
>
>
>
> On Thu, Aug 23, 2018 at 8:04 AM, Gandhi Rajan Natarajan <
> Gandhi.Natarajan@arisglobal.com> wrote:
>
> Hi Ory,
>
>
>
> I suggest you to have a look at this ctakes rest module -
> https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes-
> web-rest
>
>
>
> The custom dictionary XML needs to be placed in
> ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and
> ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as
> follows:
>
>
>
> add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/
> dictionary/lookup/fast/customDictionary.xml
>
>
>
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Wednesday, August 22, 2018 8:42 PM
>
>
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
>
>
> Thanks, Timothy.
>
> I fixed the permissions issue and was able to progress further.
>
> I am now stuck at the very last step (it would seem), the Dictionary
> Creator GUI page says this:
>
> ---
>
> Once a new dictionary has been built, point to it in one of 2 ways:
>
> Set the fast dictionary parameter LookupXml to
> org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
>
> ---
>
>
>
> Where is this LookupXml parameter to be found/added?
>
>
>
>
>
>
>
> On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <
> Timothy.Miller@childrens.harvard.edu> wrote:
>
> Looks like maybe you don't have permissions to write to that directory
> (/usr/local/). Can you write it somewhere else?
>
>
> ------------------------------
>
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Wednesday, August 22, 2018 9:15 AM
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
>
>
> Thanks a lot Ghandi, Michael, for quick and helpful replies.
>
> The video especially was very useful, but I'm encountering an error at the
> last part (building the actual dictionary).
>
> It looks like it has a problem with an HsqlDB table (log output below).
>
> Can anyone help with this? Do I need to install HsqlDB, or does it come as
> part of UMLS/Metamorphosys?
>
> Thanks,
>
> Ory
>
>
>
>
>
> 22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts:
> 450236 Texts: 1380174
>
> 22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD
> ...
>
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts:
> 1162474
>
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
>
> 22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to
> jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/
> org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
>
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
>
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
>
> 22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or
> object not found: NCI_NCI in statement [insert into nci_nci_hl7
> (CUI,NCI_NCI-HL7) values (?,?)]
>
> 22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not
> be built in /usr/local/apache-ctakes-4.0.0
>
>
>
>
>
> On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>
> wrote:
>
> Ory,
>
>
>
> In response to Gandhi's comments, the video below outlines custom
> dictionary creation in detail:
>
>
>
> https://www.youtube.com/watch?v=4aOnafv-NQs
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
>
>
>
> Best,
>
>
>
> Mike
>
>
>
>
>
>
>
> On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <
> Gandhi.Natarajan@arisglobal.com> wrote:
>
> Hi Ory,
>
> I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you
> want to lookup from other dictionaries like MEDDRA etc. , you may have to
> create your custom dictionary using cTAKES dictionary generator GUI. That’s
> what I did to include MEDDRA dictionary terms.
>
>
> -----Original Message-----
> From: Ory Henn <or...@trialjectory.com>
> Sent: Monday, August 20, 2018 1:52 PM
> To: user@ctakes.apache.org
> Cc: Guy Gildor <gu...@trialjectory.com>
> Subject: How do I add a dictionary (like NCI) to cTakes lookup?
>
> Hello,
> New user here (-;
> I've downloaded and installed cTakes (+UMLS + all resources), and am
> trying to parse a single document.
> I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
>
> 1. What is the way to make cTakes look in more UMLS dictionaries? I
> specifically need NCI as well.
> 2. Is there an easy way to make cTakes look in all UMLS dictionaries?
> Thanks,
> Ory
>
> --
>
> <https://trialmatch.me
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
> >www.trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=>
> <https://trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
> >
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
>
>
>
>
> --
>
> [image: MetiStream Logo - 500]
>
> Mike Trepanier| Big Data Engineer | MetiStream, Inc. |
> mike@metistream.com | 845 - 270 - 3129 (m) | www.metistream.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
>
>
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
> www.trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
>
>
>
>
>
> <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
>
>
>
>
> <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
RE: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Gandhi Rajan Natarajan <Ga...@arisglobal.com>.
Hi Ory,
Piper files are equivalent of the XML descriptor files which helps in adding required analysis engines.
You can manipulate analysis engines for CPE using CPE configurator.
Check this link for more info - http://uima.apache.org/downloads/releaseDocs/2.3.0-incubating/docs/html/tutorials_and_users_guides/tutorials_and_users_guides.html#ugr.tug.cpe.configurator_and_viewer
Regards,
Gandhi
From: Ory Henn <or...@trialjectory.com>
Sent: Sunday, August 26, 2018 5:25 PM
To: user@ctakes.apache.org
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks, Gandhi.
I took a look at the module, but am still a bit confused, as the connection between the CPE tool and the piper files is not clear to me.
I've succeeded in using my own dictionary with the Default Clinical Pipeline, both through -l argument and through adding a LookupXml parameter in the DefaultFastPipeline.piper script (as in ctakes-rest-service above).
But when I run the CPE tool, it still references the default dictionary.
My question is - what is the link betwenn CVD/CPE tools and the piper files? Specifically, can I configure the CPE tool to use my piper file, or configure it to use a different lookup?
Many thanks,
Ory
On Thu, Aug 23, 2018 at 8:04 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I suggest you to have a look at this ctakes rest module - https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes-web-rest
The custom dictionary XML needs to be placed in ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as follows:
add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/dictionary/lookup/fast/customDictionary.xml
From: Ory Henn <or...@trialjectory.com>>
Sent: Wednesday, August 22, 2018 8:42 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks, Timothy.
I fixed the permissions issue and was able to progress further.
I am now stuck at the very last step (it would seem), the Dictionary Creator GUI page says this:
---
Once a new dictionary has been built, point to it in one of 2 ways:
Set the fast dictionary parameter LookupXml to org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
---
Where is this LookupXml parameter to be found/added?
On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <Ti...@childrens.harvard.edu>> wrote:
Looks like maybe you don't have permissions to write to that directory (/usr/local/). Can you write it somewhere else?
________________________________
From: Ory Henn <or...@trialjectory.com>>
Sent: Wednesday, August 22, 2018 9:15 AM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks a lot Ghandi, Michael, for quick and helpful replies.
The video especially was very useful, but I'm encountering an error at the last part (building the actual dictionary).
It looks like it has a problem with an HsqlDB table (log output below).
Can anyone help with this? Do I need to install HsqlDB, or does it come as part of UMLS/Metamorphosys?
Thanks,
Ory
22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts: 450236 Texts: 1380174
22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD ...
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts: 1162474
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or object not found: NCI_NCI in statement [insert into nci_nci_hl7 (CUI,NCI_NCI-HL7) values (?,?)]
22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not be built in /usr/local/apache-ctakes-4.0.0
On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>> wrote:
Ory,
In response to Gandhi's comments, the video below outlines custom dictionary creation in detail:
https://www.youtube.com/watch?v=4aOnafv-NQs<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
Best,
Mike
On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you want to lookup from other dictionaries like MEDDRA etc. , you may have to create your custom dictionary using cTAKES dictionary generator GUI. That’s what I did to include MEDDRA dictionary terms.
-----Original Message-----
From: Ory Henn <or...@trialjectory.com>>
Sent: Monday, August 20, 2018 1:52 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Cc: Guy Gildor <gu...@trialjectory.com>>
Subject: How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=> <https://trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
--
[MetiStream Logo - 500]
Mike Trepanier| Big Data Engineer | MetiStream, Inc. | mike@metistream.com<ma...@metistream.com> | 845 - 270 - 3129 (m) | www.metistream.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Ory Henn <or...@trialjectory.com>.
Thanks, Gandhi.
I took a look at the module, but am still a bit confused, as the connection
between the CPE tool and the piper files is not clear to me.
I've succeeded in using my own dictionary with the Default Clinical
Pipeline, both through -l argument and through adding a LookupXml parameter
in the DefaultFastPipeline.piper script (as in ctakes-rest-service above).
But when I run the CPE tool, it still references the default dictionary.
My question is - what is the link betwenn CVD/CPE tools and the piper
files? Specifically, can I configure the CPE tool to use my piper file, or
configure it to use a different lookup?
Many thanks,
Ory
On Thu, Aug 23, 2018 at 8:04 AM, Gandhi Rajan Natarajan <
Gandhi.Natarajan@arisglobal.com> wrote:
> Hi Ory,
>
>
>
> I suggest you to have a look at this ctakes rest module -
> https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes-
> web-rest
>
>
>
> The custom dictionary XML needs to be placed in
> ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and
> ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as
> follows:
>
>
>
> add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/
> dictionary/lookup/fast/customDictionary.xml
>
>
>
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Wednesday, August 22, 2018 8:42 PM
>
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
>
>
> Thanks, Timothy.
>
> I fixed the permissions issue and was able to progress further.
>
> I am now stuck at the very last step (it would seem), the Dictionary
> Creator GUI page says this:
>
> ---
>
> Once a new dictionary has been built, point to it in one of 2 ways:
>
> Set the fast dictionary parameter LookupXml to
> org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
>
> ---
>
>
>
> Where is this LookupXml parameter to be found/added?
>
>
>
>
>
>
>
> On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <
> Timothy.Miller@childrens.harvard.edu> wrote:
>
> Looks like maybe you don't have permissions to write to that directory
> (/usr/local/). Can you write it somewhere else?
>
>
> ------------------------------
>
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Wednesday, August 22, 2018 9:15 AM
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
>
>
> Thanks a lot Ghandi, Michael, for quick and helpful replies.
>
> The video especially was very useful, but I'm encountering an error at the
> last part (building the actual dictionary).
>
> It looks like it has a problem with an HsqlDB table (log output below).
>
> Can anyone help with this? Do I need to install HsqlDB, or does it come as
> part of UMLS/Metamorphosys?
>
> Thanks,
>
> Ory
>
>
>
>
>
> 22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts:
> 450236 Texts: 1380174
>
> 22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD
> ...
>
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts:
> 1162474
>
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
>
> 22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to
> jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/
> org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
>
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
>
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
>
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
>
> 22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or
> object not found: NCI_NCI in statement [insert into nci_nci_hl7
> (CUI,NCI_NCI-HL7) values (?,?)]
>
> 22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not
> be built in /usr/local/apache-ctakes-4.0.0
>
>
>
>
>
> On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>
> wrote:
>
> Ory,
>
>
>
> In response to Gandhi's comments, the video below outlines custom
> dictionary creation in detail:
>
>
>
> https://www.youtube.com/watch?v=4aOnafv-NQs
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
>
>
>
> Best,
>
>
>
> Mike
>
>
>
>
>
>
>
> On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <
> Gandhi.Natarajan@arisglobal.com> wrote:
>
> Hi Ory,
>
> I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you
> want to lookup from other dictionaries like MEDDRA etc. , you may have to
> create your custom dictionary using cTAKES dictionary generator GUI. That’s
> what I did to include MEDDRA dictionary terms.
>
>
> -----Original Message-----
> From: Ory Henn <or...@trialjectory.com>
> Sent: Monday, August 20, 2018 1:52 PM
> To: user@ctakes.apache.org
> Cc: Guy Gildor <gu...@trialjectory.com>
> Subject: How do I add a dictionary (like NCI) to cTakes lookup?
>
> Hello,
> New user here (-;
> I've downloaded and installed cTakes (+UMLS + all resources), and am
> trying to parse a single document.
> I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
>
> 1. What is the way to make cTakes look in more UMLS dictionaries? I
> specifically need NCI as well.
> 2. Is there an easy way to make cTakes look in all UMLS dictionaries?
> Thanks,
> Ory
>
> --
>
> <https://trialmatch.me
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
> >www.trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=>
> <https://trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
> >
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
>
>
>
>
> --
>
> [image: MetiStream Logo - 500]
>
> Mike Trepanier| Big Data Engineer | MetiStream, Inc. |
> mike@metistream.com | 845 - 270 - 3129 (m) | www.metistream.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
>
>
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
> www.trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
>
>
>
>
>
> <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
RE: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Gandhi Rajan Natarajan <Ga...@arisglobal.com>.
Hi Ory,
I suggest you to have a look at this ctakes rest module - https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes-web-rest
The custom dictionary XML needs to be placed in ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as follows:
add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/dictionary/lookup/fast/customDictionary.xml
From: Ory Henn <or...@trialjectory.com>
Sent: Wednesday, August 22, 2018 8:42 PM
To: user@ctakes.apache.org
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks, Timothy.
I fixed the permissions issue and was able to progress further.
I am now stuck at the very last step (it would seem), the Dictionary Creator GUI page says this:
---
Once a new dictionary has been built, point to it in one of 2 ways:
Set the fast dictionary parameter LookupXml to org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
---
Where is this LookupXml parameter to be found/added?
On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <Ti...@childrens.harvard.edu>> wrote:
Looks like maybe you don't have permissions to write to that directory (/usr/local/). Can you write it somewhere else?
________________________________
From: Ory Henn <or...@trialjectory.com>>
Sent: Wednesday, August 22, 2018 9:15 AM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks a lot Ghandi, Michael, for quick and helpful replies.
The video especially was very useful, but I'm encountering an error at the last part (building the actual dictionary).
It looks like it has a problem with an HsqlDB table (log output below).
Can anyone help with this? Do I need to install HsqlDB, or does it come as part of UMLS/Metamorphosys?
Thanks,
Ory
22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts: 450236 Texts: 1380174
22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD ...
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts: 1162474
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or object not found: NCI_NCI in statement [insert into nci_nci_hl7 (CUI,NCI_NCI-HL7) values (?,?)]
22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not be built in /usr/local/apache-ctakes-4.0.0
On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>> wrote:
Ory,
In response to Gandhi's comments, the video below outlines custom dictionary creation in detail:
https://www.youtube.com/watch?v=4aOnafv-NQs<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
Best,
Mike
On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you want to lookup from other dictionaries like MEDDRA etc. , you may have to create your custom dictionary using cTAKES dictionary generator GUI. That’s what I did to include MEDDRA dictionary terms.
-----Original Message-----
From: Ory Henn <or...@trialjectory.com>>
Sent: Monday, August 20, 2018 1:52 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Cc: Guy Gildor <gu...@trialjectory.com>>
Subject: How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=> <https://trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
--
[MetiStream Logo - 500]
Mike Trepanier| Big Data Engineer | MetiStream, Inc. | mike@metistream.com<ma...@metistream.com> | 845 - 270 - 3129 (m) | www.metistream.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://trialmatch.me>www.trialjectory.com<https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by Ory Henn <or...@trialjectory.com>.
Thanks, Timothy.
I fixed the permissions issue and was able to progress further.
I am now stuck at the very last step (it would seem), the Dictionary
Creator GUI page says this:
---
Once a new dictionary has been built, point to it in one of 2 ways:
Set the fast dictionary parameter LookupXml to
org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml
---
Where is this LookupXml parameter to be found/added?
On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy <
Timothy.Miller@childrens.harvard.edu> wrote:
> Looks like maybe you don't have permissions to write to that directory
> (/usr/local/). Can you write it somewhere else?
>
>
> ------------------------------
> *From:* Ory Henn <or...@trialjectory.com>
> *Sent:* Wednesday, August 22, 2018 9:15 AM
> *To:* user@ctakes.apache.org
> *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup?
> [EXTERNAL]
>
> Thanks a lot Ghandi, Michael, for quick and helpful replies.
> The video especially was very useful, but I'm encountering an error at the
> last part (building the actual dictionary).
> It looks like it has a problem with an HsqlDB table (log output below).
> Can anyone help with this? Do I need to install HsqlDB, or does it come as
> part of UMLS/Metamorphosys?
> Thanks,
> Ory
>
>
> 22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts:
> 450236 Texts: 1380174
> 22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD
> ...
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts:
> 1162474
> 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
> 22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to
> jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/
> org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
> 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
> 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
> 22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or
> object not found: NCI_NCI in statement [insert into nci_nci_hl7
> (CUI,NCI_NCI-HL7) values (?,?)]
> 22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not
> be built in /usr/local/apache-ctakes-4.0.0
>
>
> On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>
> wrote:
>
>> Ory,
>>
>> In response to Gandhi's comments, the video below outlines custom
>> dictionary creation in detail:
>>
>> https://www.youtube.com/watch?v=4aOnafv-NQs
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
>>
>> Best,
>>
>> Mike
>>
>>
>>
>> On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <
>> Gandhi.Natarajan@arisglobal.com> wrote:
>>
>>> Hi Ory,
>>>
>>> I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you
>>> want to lookup from other dictionaries like MEDDRA etc. , you may have to
>>> create your custom dictionary using cTAKES dictionary generator GUI. That’s
>>> what I did to include MEDDRA dictionary terms.
>>>
>>> -----Original Message-----
>>> From: Ory Henn <or...@trialjectory.com>
>>> Sent: Monday, August 20, 2018 1:52 PM
>>> To: user@ctakes.apache.org
>>> Cc: Guy Gildor <gu...@trialjectory.com>
>>> Subject: How do I add a dictionary (like NCI) to cTakes lookup?
>>>
>>> Hello,
>>> New user here (-;
>>> I've downloaded and installed cTakes (+UMLS + all resources), and am
>>> trying to parse a single document.
>>> I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and
>>> SNOMED_CT.
>>>
>>> 1. What is the way to make cTakes look in more UMLS dictionaries? I
>>> specifically need NCI as well.
>>> 2. Is there an easy way to make cTakes look in all UMLS dictionaries?
>>> Thanks,
>>> Ory
>>>
>>> --
>>>
>>> <https://trialmatch.me
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
>>> >www.trialjectory.com
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=>
>>> <https://trialjectory.com
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
>>> >
>>> This email and any files transmitted with it are confidential and
>>> intended solely for the use of the individual or entity to whom they are
>>> addressed. If you are not the named addressee you should not disseminate,
>>> distribute or copy this e-mail. Please notify the sender or system manager
>>> by email immediately if you have received this e-mail by mistake and delete
>>> this e-mail from your system. If you are not the intended recipient you are
>>> notified that disclosing, copying, distributing or taking any action in
>>> reliance on the contents of this information is strictly prohibited and
>>> against the law.
>>>
>>
>>
>>
>> --
>> [image: MetiStream Logo - 500]
>> Mike Trepanier| Big Data Engineer | MetiStream, Inc. |
>> mike@metistream.com | 845 - 270 - 3129 (m) | www.metistream.com
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
>>
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>
> www.trialjectory.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
>
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Posted by "Miller, Timothy" <Ti...@childrens.harvard.edu>.
Looks like maybe you don't have permissions to write to that directory (/usr/local/). Can you write it somewhere else?
________________________________
From: Ory Henn <or...@trialjectory.com>
Sent: Wednesday, August 22, 2018 9:15 AM
To: user@ctakes.apache.org
Subject: Re: How do I add a dictionary (like NCI) to cTakes lookup? [EXTERNAL]
Thanks a lot Ghandi, Michael, for quick and helpful replies.
The video especially was very useful, but I'm encountering an error at the last part (building the actual dictionary).
It looks like it has a problem with an HsqlDB table (log output below).
Can anyone help with this? Do I need to install HsqlDB, or does it come as part of UMLS/Metamorphosys?
Thanks,
Ory
22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts: 450236 Texts: 1380174
22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD ...
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts: 1162474
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or object not found: NCI_NCI in statement [insert into nci_nci_hl7 (CUI,NCI_NCI-HL7) values (?,?)]
22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not be built in /usr/local/apache-ctakes-4.0.0
On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>> wrote:
Ory,
In response to Gandhi's comments, the video below outlines custom dictionary creation in detail:
https://www.youtube.com/watch?v=4aOnafv-NQs<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=>
Best,
Mike
On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <Ga...@arisglobal.com>> wrote:
Hi Ory,
I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you want to lookup from other dictionaries like MEDDRA etc. , you may have to create your custom dictionary using cTAKES dictionary generator GUI. That’s what I did to include MEDDRA dictionary terms.
-----Original Message-----
From: Ory Henn <or...@trialjectory.com>>
Sent: Monday, August 20, 2018 1:52 PM
To: user@ctakes.apache.org<ma...@ctakes.apache.org>
Cc: Guy Gildor <gu...@trialjectory.com>>
Subject: How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=> <https://trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.
--
[MetiStream Logo - 500]
Mike Trepanier| Big Data Engineer | MetiStream, Inc. | mike@metistream.com<ma...@metistream.com> | 845 - 270 - 3129 (m) | www.metistream.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=>
[https://res.cloudinary.com/dyswk0o6o/image/upload/v1525083591/logo4email_smaller_f5pqgv.png]
<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=>www.trialjectory.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=>
Re: How do I add a dictionary (like NCI) to cTakes lookup?
Posted by Ory Henn <or...@trialjectory.com>.
Thanks a lot Ghandi, Michael, for quick and helpful replies.
The video especially was very useful, but I'm encountering an error at the
last part (building the actual dictionary).
It looks like it has a problem with an HsqlDB table (log output below).
Can anyone help with this? Do I need to install HsqlDB, or does it come as
part of UMLS/Metamorphosys?
Thanks,
Ory
22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts:
450236 Texts: 1380174
22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD ...
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts:
1162474
22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts
22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to
jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci
as SA
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done
22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end
22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1
22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or
object not found: NCI_NCI in statement [insert into nci_nci_hl7
(CUI,NCI_NCI-HL7) values (?,?)]
22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not
be built in /usr/local/apache-ctakes-4.0.0
On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <mi...@metistream.com>
wrote:
> Ory,
>
> In response to Gandhi's comments, the video below outlines custom
> dictionary creation in detail:
>
> https://www.youtube.com/watch?v=4aOnafv-NQs
>
> Best,
>
> Mike
>
>
>
> On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <
> Gandhi.Natarajan@arisglobal.com> wrote:
>
>> Hi Ory,
>>
>> I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you
>> want to lookup from other dictionaries like MEDDRA etc. , you may have to
>> create your custom dictionary using cTAKES dictionary generator GUI. That’s
>> what I did to include MEDDRA dictionary terms.
>>
>> -----Original Message-----
>> From: Ory Henn <or...@trialjectory.com>
>> Sent: Monday, August 20, 2018 1:52 PM
>> To: user@ctakes.apache.org
>> Cc: Guy Gildor <gu...@trialjectory.com>
>> Subject: How do I add a dictionary (like NCI) to cTakes lookup?
>>
>> Hello,
>> New user here (-;
>> I've downloaded and installed cTakes (+UMLS + all resources), and am
>> trying to parse a single document.
>> I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and
>> SNOMED_CT.
>>
>> 1. What is the way to make cTakes look in more UMLS dictionaries? I
>> specifically need NCI as well.
>> 2. Is there an easy way to make cTakes look in all UMLS dictionaries?
>> Thanks,
>> Ory
>>
>> --
>>
>> <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
>> This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity to whom they are
>> addressed. If you are not the named addressee you should not disseminate,
>> distribute or copy this e-mail. Please notify the sender or system manager
>> by email immediately if you have received this e-mail by mistake and delete
>> this e-mail from your system. If you are not the intended recipient you are
>> notified that disclosing, copying, distributing or taking any action in
>> reliance on the contents of this information is strictly prohibited and
>> against the law.
>>
>
>
>
> --
> [image: MetiStream Logo - 500]
> Mike Trepanier| Big Data Engineer | MetiStream, Inc. |
> mike@metistream.com | 845 - 270 - 3129 (m) | www.metistream.com
>
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
Re: How do I add a dictionary (like NCI) to cTakes lookup?
Posted by Michael Trepanier <mi...@metistream.com>.
Ory,
In response to Gandhi's comments, the video below outlines custom
dictionary creation in detail:
https://www.youtube.com/watch?v=4aOnafv-NQs
Best,
Mike
On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan <
Gandhi.Natarajan@arisglobal.com> wrote:
> Hi Ory,
>
> I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you
> want to lookup from other dictionaries like MEDDRA etc. , you may have to
> create your custom dictionary using cTAKES dictionary generator GUI. That’s
> what I did to include MEDDRA dictionary terms.
>
> -----Original Message-----
> From: Ory Henn <or...@trialjectory.com>
> Sent: Monday, August 20, 2018 1:52 PM
> To: user@ctakes.apache.org
> Cc: Guy Gildor <gu...@trialjectory.com>
> Subject: How do I add a dictionary (like NCI) to cTakes lookup?
>
> Hello,
> New user here (-;
> I've downloaded and installed cTakes (+UMLS + all resources), and am
> trying to parse a single document.
> I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
>
> 1. What is the way to make cTakes look in more UMLS dictionaries? I
> specifically need NCI as well.
> 2. Is there an easy way to make cTakes look in all UMLS dictionaries?
> Thanks,
> Ory
>
> --
>
> <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender or system manager by email
> immediately if you have received this e-mail by mistake and delete this
> e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited and
> against the law.
>
--
[image: MetiStream Logo - 500]
Mike Trepanier| Big Data Engineer | MetiStream, Inc. | mike@metistream.com |
845 - 270 - 3129 (m) | www.metistream.com
RE: How do I add a dictionary (like NCI) to cTakes lookup?
Posted by Gandhi Rajan Natarajan <Ga...@arisglobal.com>.
Hi Ory,
I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you want to lookup from other dictionaries like MEDDRA etc. , you may have to create your custom dictionary using cTAKES dictionary generator GUI. That’s what I did to include MEDDRA dictionary terms.
-----Original Message-----
From: Ory Henn <or...@trialjectory.com>
Sent: Monday, August 20, 2018 1:52 PM
To: user@ctakes.apache.org
Cc: Guy Gildor <gu...@trialjectory.com>
Subject: How do I add a dictionary (like NCI) to cTakes lookup?
Hello,
New user here (-;
I've downloaded and installed cTakes (+UMLS + all resources), and am trying to parse a single document.
I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT.
1. What is the way to make cTakes look in more UMLS dictionaries? I specifically need NCI as well.
2. Is there an easy way to make cTakes look in all UMLS dictionaries?
Thanks,
Ory
--
<https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender or system manager by email immediately if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited and against the law.