You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by "Mullane, Sean *HS" <SP...@hscmail.mcc.virginia.edu> on 2017/06/02 22:03:35 UTC
ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean
RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
Hi Sean,
That is great - thank you for sharing your solution!
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Tuesday, June 06, 2017 11:19 AM
To: Finan, Sean; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Sean,
Thanks for the information. It's good to have an alternative to try. I was finally able to make the DBConsumer do what I needed. In case it helps anyone else:
The DBConsumer component page on the wiki helped me figure out what to do. The reason the CUIs weren't being written to the database is they were subordinate to other identified annotations (e.g. DiseaseDisorderMention) which were not being written either. The wiki mentioned that those can be added to the output by adding a table for each UIMA type in the output containing the appropriate fields for that type, and also a row needs to be added to the ref_type_uima table referencing the new table. Once I added the other identified annotation types like this the DBConsumer started populating them and the (already existing) anno_ontology_concept table too.
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 10:29 AM
To: Mullane, Sean *HS; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I have used JdbcWriterTemplate in some private projects using i2b2 https://urldefense.proofpoint.com/v2/url?u=https-3A__www.i2b2.org_&d=DwIFAg&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=-S2yhMPnH7V386vxUmWnkbJf2az3CKeW4UrGctDZ5pM&s=oFeuIdWHLZeCIOJ4znIdAdGtR0IP0NOnokW1ql-s6gg&e= The template is actually set up to write to an i2b2-style table by default.
Unfortunately I cannot provide specifics on those configurations, but if you look at JdbcWriterTemplate.xml in ctakes-core's desc/cas_consumer/ you should get an idea of what parameter values you need to set from the <!-- CHANGE ME --> comments.
Creating your own copy of that xml descriptor and adding it at the end of a pipeline descriptor should do the trick. If you want to use a piper file then just add those parameter names and values indicated in the descriptor xml using piper commands in a copy of the DefaultFastPipeline.piper in ctakes-clinical-pipeline-res , adding JdbcWriterTemplate to the end of the pipeline.
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_Piper-2BFiles&d=DwIFAg&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=-S2yhMPnH7V386vxUmWnkbJf2az3CKeW4UrGctDZ5pM&s=jcEXfU62LXbJHPqHpeV6otstuGywbIV8drh-_Be4Sj0&e=
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Monday, June 05, 2017 10:06 AM
To: Finan, Sean; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Do you know of any example code using JdbcWriterTemplate? I'm loathe to abandon a setup that has worked for me in the past but example code would help.
Thanks,
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 9:03 AM
To: dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I am unfamiliar with all things ytex, but a cursory glance at that class makes me think that it is only meant to write an XMI file.
You can have a look at JdbcWriterTemplate and its parent AbstractJdbcWriter in ctakes core.cc, which can write to an sql database.
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Friday, June 02, 2017 6:04 PM
To: dev@ctakes.apache.org
Subject: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean
RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Posted by "Mullane, Sean *HS" <SP...@hscmail.mcc.virginia.edu>.
Sean,
Thanks for the information. It's good to have an alternative to try. I was finally able to make the DBConsumer do what I needed. In case it helps anyone else:
The DBConsumer component page on the wiki helped me figure out what to do. The reason the CUIs weren't being written to the database is they were subordinate to other identified annotations (e.g. DiseaseDisorderMention) which were not being written either. The wiki mentioned that those can be added to the output by adding a table for each UIMA type in the output containing the appropriate fields for that type, and also a row needs to be added to the ref_type_uima table referencing the new table. Once I added the other identified annotation types like this the DBConsumer started populating them and the (already existing) anno_ontology_concept table too.
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 10:29 AM
To: Mullane, Sean *HS; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I have used JdbcWriterTemplate in some private projects using i2b2 https://www.i2b2.org/ The template is actually set up to write to an i2b2-style table by default.
Unfortunately I cannot provide specifics on those configurations, but if you look at JdbcWriterTemplate.xml in ctakes-core's desc/cas_consumer/ you should get an idea of what parameter values you need to set from the <!-- CHANGE ME --> comments.
Creating your own copy of that xml descriptor and adding it at the end of a pipeline descriptor should do the trick. If you want to use a piper file then just add those parameter names and values indicated in the descriptor xml using piper commands in a copy of the DefaultFastPipeline.piper in ctakes-clinical-pipeline-res , adding JdbcWriterTemplate to the end of the pipeline.
https://cwiki.apache.org/confluence/display/CTAKES/Piper+Files
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Monday, June 05, 2017 10:06 AM
To: Finan, Sean; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Do you know of any example code using JdbcWriterTemplate? I'm loathe to abandon a setup that has worked for me in the past but example code would help.
Thanks,
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 9:03 AM
To: dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I am unfamiliar with all things ytex, but a cursory glance at that class makes me think that it is only meant to write an XMI file.
You can have a look at JdbcWriterTemplate and its parent AbstractJdbcWriter in ctakes core.cc, which can write to an sql database.
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Friday, June 02, 2017 6:04 PM
To: dev@ctakes.apache.org
Subject: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean
RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
Hi Sean,
I have used JdbcWriterTemplate in some private projects using i2b2 https://www.i2b2.org/ The template is actually set up to write to an i2b2-style table by default.
Unfortunately I cannot provide specifics on those configurations, but if you look at JdbcWriterTemplate.xml in ctakes-core's desc/cas_consumer/ you should get an idea of what parameter values you need to set from the <!-- CHANGE ME --> comments.
Creating your own copy of that xml descriptor and adding it at the end of a pipeline descriptor should do the trick. If you want to use a piper file then just add those parameter names and values indicated in the descriptor xml using piper commands in a copy of the DefaultFastPipeline.piper in ctakes-clinical-pipeline-res , adding JdbcWriterTemplate to the end of the pipeline.
https://cwiki.apache.org/confluence/display/CTAKES/Piper+Files
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Monday, June 05, 2017 10:06 AM
To: Finan, Sean; dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Do you know of any example code using JdbcWriterTemplate? I'm loathe to abandon a setup that has worked for me in the past but example code would help.
Thanks,
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 9:03 AM
To: dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I am unfamiliar with all things ytex, but a cursory glance at that class makes me think that it is only meant to write an XMI file.
You can have a look at JdbcWriterTemplate and its parent AbstractJdbcWriter in ctakes core.cc, which can write to an sql database.
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Friday, June 02, 2017 6:04 PM
To: dev@ctakes.apache.org
Subject: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean
RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Posted by "Mullane, Sean *HS" <SP...@hscmail.mcc.virginia.edu>.
Do you know of any example code using JdbcWriterTemplate? I'm loathe to abandon a setup that has worked for me in the past but example code would help.
Thanks,
Sean
-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Monday, June 05, 2017 9:03 AM
To: dev@ctakes.apache.org
Subject: RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Hi Sean,
I am unfamiliar with all things ytex, but a cursory glance at that class makes me think that it is only meant to write an XMI file.
You can have a look at JdbcWriterTemplate and its parent AbstractJdbcWriter in ctakes core.cc, which can write to an sql database.
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Friday, June 02, 2017 6:04 PM
To: dev@ctakes.apache.org
Subject: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean
RE: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
Hi Sean,
I am unfamiliar with all things ytex, but a cursory glance at that class makes me think that it is only meant to write an XMI file.
You can have a look at JdbcWriterTemplate and its parent AbstractJdbcWriter in ctakes core.cc, which can write to an sql database.
Sean
-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Friday, June 02, 2017 6:04 PM
To: dev@ctakes.apache.org
Subject: ctakes 4.0.0 YTEX DBConsumer not populating v_document_cui_sent
I am using the DBConsumer AE to write annotations to a SQL Server database, but I can't get it to write out the CUIs to the database. Trying the prettyWriter consumer does show that my pipeline (the default fast piper) is annotating the CUIs; the problem seems to be between that and the YTEX DBConsumer. Can anyone suggest how I might figure out the problem? I have been working through the debugger in Eclipse but haven't had luck so far. I can provide detailed config information upon request.
For what it's worth I have had a similar lack of success using the standard XML pipeline descriptor instead of the piper. The odd thing is that I had this working fine in 3.2.2, now when I revisit I can't get it to write CUI annotations to the database either. This leads me to believe this is a configuration issue but again, I have so far been unable to track it down.
Thanks,
Sean