You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by David Kincaid <ki...@gmail.com> on 2017/04/23 02:38:25 UTC
cTAKES 4.0rc3 - custom dictionary needs rebuilding?
I finally had some time this weekend to give 4.0.0rc3 a run. The install
and configuration instructions worked perfectly on my Linux laptop and I
was able to use CVD to run the clinical pipeline successfully against some
of the clinical notes I have here.
However, I ran into a problem when I tried running my own customized
pipeline that includes a custom dictionary that I create (to include the
SNOMED veterinary extension). I get an exception while it's loading that
the dictionary is not the correct version. So, I assume I just need to
recreate that custom database? Or is there a migration utility from the old
database version to the new version?
I don't remember seeing mention of this, but certainly could have missed
it. If it's not in the release notes or install/upgrade instructions it
probably should be there.
Thanks to everyone for pulling this release together and getting it out the
door!
- Dave
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by David Kincaid <ki...@gmail.com>.
I ended up just using the DictionaryCreator and recreating the dictionary,
so it's working now. I haven't looked at the section you added to the user
guide yet, but should have some time tonight to give it a go and test it.
Thanks,
Dave
On Mon, Apr 24, 2017 at 10:29 AM, James Masanz <ma...@gmail.com>
wrote:
> Dave,
> Is there something we can clarify on how to create the URL?
>
> On Sun, Apr 23, 2017 at 3:23 PM, David Kincaid <ki...@gmail.com>
> wrote:
>
> > I got it figured out. I was using the wrong URL path for the database.
> > Working great now!
> >
> > - Dave
> >
> > On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid <ki...@gmail.com>
> > wrote:
> >
> > > Thanks for the reply, James. I'm not very familiar with HSQLDB, so I
> may
> > > be doing it wrong, but I launched the GUI using "java
> > > -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing
> > the
> > > "Type" to "HSQL Database Engine Standalone" and then setting the URL to
> > > point to my custom dictionary directory. It throws a SQLException that
> > says
> > > "wrong database file version". Is there some other step I'm missing?
> > >
> > > - Dave
> > >
> > > On Sat, Apr 22, 2017 at 10:31 PM, James Masanz <masanz.james@gmail.com
> >
> > > wrote:
> > >
> > >> Dave,
> > >>
> > >> It's an oversight that we didn't document that. Thanks for pointing
> that
> > >> out! I'll update the documentation tomorrow and post again with
> proper
> > >> details of the command to use.
> > >>
> > >> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
> > >> version of that jar, which is included with the cTAKES 4.0 convenience
> > >> binary, to open your database. Then you close (shutdown) your DB and
> it
> > >> gets converted to 2.3.4 for you. But if the .properties file for your
> DB
> > >> indicates it's readonly, you need to have edited the properties file
> > first
> > >> to remove that. That gives you the idea. I'll write it up properly
> > >> tomorrow.
> > >>
> > >> Thanks for testing,
> > >> -- James
> > >>
> > >>
> > >> *Sent from my phone.*
> > >>
> > >> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com>
> > wrote:
> > >>
> > >> I finally had some time this weekend to give 4.0.0rc3 a run. The
> install
> > >> and configuration instructions worked perfectly on my Linux laptop
> and I
> > >> was able to use CVD to run the clinical pipeline successfully against
> > some
> > >> of the clinical notes I have here.
> > >>
> > >> However, I ran into a problem when I tried running my own customized
> > >> pipeline that includes a custom dictionary that I create (to include
> the
> > >> SNOMED veterinary extension). I get an exception while it's loading
> that
> > >> the dictionary is not the correct version. So, I assume I just need to
> > >> recreate that custom database? Or is there a migration utility from
> the
> > >> old
> > >> database version to the new version?
> > >>
> > >> I don't remember seeing mention of this, but certainly could have
> missed
> > >> it. If it's not in the release notes or install/upgrade instructions
> it
> > >> probably should be there.
> > >>
> > >> Thanks to everyone for pulling this release together and getting it
> out
> > >> the
> > >> door!
> > >>
> > >> - Dave
> > >>
> > >
> > >
> >
>
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by James Masanz <ma...@gmail.com>.
Dave,
Is there something we can clarify on how to create the URL?
On Sun, Apr 23, 2017 at 3:23 PM, David Kincaid <ki...@gmail.com>
wrote:
> I got it figured out. I was using the wrong URL path for the database.
> Working great now!
>
> - Dave
>
> On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid <ki...@gmail.com>
> wrote:
>
> > Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may
> > be doing it wrong, but I launched the GUI using "java
> > -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing
> the
> > "Type" to "HSQL Database Engine Standalone" and then setting the URL to
> > point to my custom dictionary directory. It throws a SQLException that
> says
> > "wrong database file version". Is there some other step I'm missing?
> >
> > - Dave
> >
> > On Sat, Apr 22, 2017 at 10:31 PM, James Masanz <ma...@gmail.com>
> > wrote:
> >
> >> Dave,
> >>
> >> It's an oversight that we didn't document that. Thanks for pointing that
> >> out! I'll update the documentation tomorrow and post again with proper
> >> details of the command to use.
> >>
> >> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
> >> version of that jar, which is included with the cTAKES 4.0 convenience
> >> binary, to open your database. Then you close (shutdown) your DB and it
> >> gets converted to 2.3.4 for you. But if the .properties file for your DB
> >> indicates it's readonly, you need to have edited the properties file
> first
> >> to remove that. That gives you the idea. I'll write it up properly
> >> tomorrow.
> >>
> >> Thanks for testing,
> >> -- James
> >>
> >>
> >> *Sent from my phone.*
> >>
> >> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com>
> wrote:
> >>
> >> I finally had some time this weekend to give 4.0.0rc3 a run. The install
> >> and configuration instructions worked perfectly on my Linux laptop and I
> >> was able to use CVD to run the clinical pipeline successfully against
> some
> >> of the clinical notes I have here.
> >>
> >> However, I ran into a problem when I tried running my own customized
> >> pipeline that includes a custom dictionary that I create (to include the
> >> SNOMED veterinary extension). I get an exception while it's loading that
> >> the dictionary is not the correct version. So, I assume I just need to
> >> recreate that custom database? Or is there a migration utility from the
> >> old
> >> database version to the new version?
> >>
> >> I don't remember seeing mention of this, but certainly could have missed
> >> it. If it's not in the release notes or install/upgrade instructions it
> >> probably should be there.
> >>
> >> Thanks to everyone for pulling this release together and getting it out
> >> the
> >> door!
> >>
> >> - Dave
> >>
> >
> >
>
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by David Kincaid <ki...@gmail.com>.
Sorry about this. Actually no, it's not working. I just created a new blank
database apparently in a new directory. Still having the original problem
trying to open the DB in HSQL GUI. Any ideas?
- Dave
On Sun, Apr 23, 2017 at 2:23 PM, David Kincaid <ki...@gmail.com>
wrote:
> I got it figured out. I was using the wrong URL path for the database.
> Working great now!
>
> - Dave
>
> On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid <ki...@gmail.com>
> wrote:
>
>> Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may
>> be doing it wrong, but I launched the GUI using "java
>> -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing the
>> "Type" to "HSQL Database Engine Standalone" and then setting the URL to
>> point to my custom dictionary directory. It throws a SQLException that says
>> "wrong database file version". Is there some other step I'm missing?
>>
>> - Dave
>>
>> On Sat, Apr 22, 2017 at 10:31 PM, James Masanz <ma...@gmail.com>
>> wrote:
>>
>>> Dave,
>>>
>>> It's an oversight that we didn't document that. Thanks for pointing that
>>> out! I'll update the documentation tomorrow and post again with proper
>>> details of the command to use.
>>>
>>> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
>>> version of that jar, which is included with the cTAKES 4.0 convenience
>>> binary, to open your database. Then you close (shutdown) your DB and it
>>> gets converted to 2.3.4 for you. But if the .properties file for your DB
>>> indicates it's readonly, you need to have edited the properties file
>>> first
>>> to remove that. That gives you the idea. I'll write it up properly
>>> tomorrow.
>>>
>>> Thanks for testing,
>>> -- James
>>>
>>>
>>> *Sent from my phone.*
>>>
>>> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
>>>
>>> I finally had some time this weekend to give 4.0.0rc3 a run. The install
>>> and configuration instructions worked perfectly on my Linux laptop and I
>>> was able to use CVD to run the clinical pipeline successfully against
>>> some
>>> of the clinical notes I have here.
>>>
>>> However, I ran into a problem when I tried running my own customized
>>> pipeline that includes a custom dictionary that I create (to include the
>>> SNOMED veterinary extension). I get an exception while it's loading that
>>> the dictionary is not the correct version. So, I assume I just need to
>>> recreate that custom database? Or is there a migration utility from the
>>> old
>>> database version to the new version?
>>>
>>> I don't remember seeing mention of this, but certainly could have missed
>>> it. If it's not in the release notes or install/upgrade instructions it
>>> probably should be there.
>>>
>>> Thanks to everyone for pulling this release together and getting it out
>>> the
>>> door!
>>>
>>> - Dave
>>>
>>
>>
>
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by David Kincaid <ki...@gmail.com>.
I got it figured out. I was using the wrong URL path for the database.
Working great now!
- Dave
On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid <ki...@gmail.com>
wrote:
> Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may
> be doing it wrong, but I launched the GUI using "java
> -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing the
> "Type" to "HSQL Database Engine Standalone" and then setting the URL to
> point to my custom dictionary directory. It throws a SQLException that says
> "wrong database file version". Is there some other step I'm missing?
>
> - Dave
>
> On Sat, Apr 22, 2017 at 10:31 PM, James Masanz <ma...@gmail.com>
> wrote:
>
>> Dave,
>>
>> It's an oversight that we didn't document that. Thanks for pointing that
>> out! I'll update the documentation tomorrow and post again with proper
>> details of the command to use.
>>
>> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
>> version of that jar, which is included with the cTAKES 4.0 convenience
>> binary, to open your database. Then you close (shutdown) your DB and it
>> gets converted to 2.3.4 for you. But if the .properties file for your DB
>> indicates it's readonly, you need to have edited the properties file first
>> to remove that. That gives you the idea. I'll write it up properly
>> tomorrow.
>>
>> Thanks for testing,
>> -- James
>>
>>
>> *Sent from my phone.*
>>
>> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
>>
>> I finally had some time this weekend to give 4.0.0rc3 a run. The install
>> and configuration instructions worked perfectly on my Linux laptop and I
>> was able to use CVD to run the clinical pipeline successfully against some
>> of the clinical notes I have here.
>>
>> However, I ran into a problem when I tried running my own customized
>> pipeline that includes a custom dictionary that I create (to include the
>> SNOMED veterinary extension). I get an exception while it's loading that
>> the dictionary is not the correct version. So, I assume I just need to
>> recreate that custom database? Or is there a migration utility from the
>> old
>> database version to the new version?
>>
>> I don't remember seeing mention of this, but certainly could have missed
>> it. If it's not in the release notes or install/upgrade instructions it
>> probably should be there.
>>
>> Thanks to everyone for pulling this release together and getting it out
>> the
>> door!
>>
>> - Dave
>>
>
>
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by David Kincaid <ki...@gmail.com>.
Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may be
doing it wrong, but I launched the GUI using "java -jar hsqldb-2.3.4.jar"
and then tried to connect to the DB by changing the "Type" to "HSQL
Database Engine Standalone" and then setting the URL to point to my custom
dictionary directory. It throws a SQLException that says "wrong database
file version". Is there some other step I'm missing?
- Dave
On Sat, Apr 22, 2017 at 10:31 PM, James Masanz <ma...@gmail.com>
wrote:
> Dave,
>
> It's an oversight that we didn't document that. Thanks for pointing that
> out! I'll update the documentation tomorrow and post again with proper
> details of the command to use.
>
> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
> version of that jar, which is included with the cTAKES 4.0 convenience
> binary, to open your database. Then you close (shutdown) your DB and it
> gets converted to 2.3.4 for you. But if the .properties file for your DB
> indicates it's readonly, you need to have edited the properties file first
> to remove that. That gives you the idea. I'll write it up properly
> tomorrow.
>
> Thanks for testing,
> -- James
>
>
> *Sent from my phone.*
>
> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
>
> I finally had some time this weekend to give 4.0.0rc3 a run. The install
> and configuration instructions worked perfectly on my Linux laptop and I
> was able to use CVD to run the clinical pipeline successfully against some
> of the clinical notes I have here.
>
> However, I ran into a problem when I tried running my own customized
> pipeline that includes a custom dictionary that I create (to include the
> SNOMED veterinary extension). I get an exception while it's loading that
> the dictionary is not the correct version. So, I assume I just need to
> recreate that custom database? Or is there a migration utility from the old
> database version to the new version?
>
> I don't remember seeing mention of this, but certainly could have missed
> it. If it's not in the release notes or install/upgrade instructions it
> probably should be there.
>
> Thanks to everyone for pulling this release together and getting it out the
> door!
>
> - Dave
>
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by James Masanz <ma...@gmail.com>.
I've added a section[1] to the User Install Guide describing how to convert
an HSQLDB from hsqldb 1.8 to 2.3.4 for cTAKES 4.0
[1]
https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+4.0+User+Install+Guide#cTAKES4.0UserInstallGuide-ConvertDictionariesYou'vePreviouslyCreatedtobeCompatiblewithcTAKES4.0
On Sun, Apr 23, 2017 at 10:03 AM, Finan, Sean <
Sean.Finan@childrens.harvard.edu> wrote:
> Thank you James - great explanation and information.
>
> Hi David,
> There is a new version of the dictionary creator gui distributed with
> ctakes 4.0. It is 99% the same as the version in sandbox. One important
> difference is that it now produces a database in hsqldb 2.3.4 - compatible
> with ctakes 4.0.
> Another difference (important to you) is that there is an exclusion list
> in a data file that lists cui / term combinations that can be excluded. By
> default the medications "toDAY" and "ToMORROW" are in that list. It is
> obviously done to prevent the frequent false positives that you spoke of in
> your Amia presentation. For non-vet use these exclusions are pretty
> valid. Since you are interested in a vet dictionary fi you use the new
> dictionary creator you can decide whether or not you want them to be
> included.
>
> Sean
>
> -----Original Message-----
> From: James Masanz [mailto:masanz.james@gmail.com]
> Sent: Saturday, April 22, 2017 11:32 PM
> To: dev@ctakes.apache.org
> Subject: Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
>
> Dave,
>
> It's an oversight that we didn't document that. Thanks for pointing that
> out! I'll update the documentation tomorrow and post again with proper
> details of the command to use.
>
> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
> version of that jar, which is included with the cTAKES 4.0 convenience
> binary, to open your database. Then you close (shutdown) your DB and it
> gets converted to 2.3.4 for you. But if the .properties file for your DB
> indicates it's readonly, you need to have edited the properties file first
> to remove that. That gives you the idea. I'll write it up properly tomorrow.
>
> Thanks for testing,
> -- James
>
>
> *Sent from my phone.*
>
> On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
>
> I finally had some time this weekend to give 4.0.0rc3 a run. The install
> and configuration instructions worked perfectly on my Linux laptop and I
> was able to use CVD to run the clinical pipeline successfully against some
> of the clinical notes I have here.
>
> However, I ran into a problem when I tried running my own customized
> pipeline that includes a custom dictionary that I create (to include the
> SNOMED veterinary extension). I get an exception while it's loading that
> the dictionary is not the correct version. So, I assume I just need to
> recreate that custom database? Or is there a migration utility from the old
> database version to the new version?
>
> I don't remember seeing mention of this, but certainly could have missed
> it. If it's not in the release notes or install/upgrade instructions it
> probably should be there.
>
> Thanks to everyone for pulling this release together and getting it out
> the door!
>
> - Dave
>
RE: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by "Finan, Sean" <Se...@childrens.harvard.edu>.
Thank you James - great explanation and information.
Hi David,
There is a new version of the dictionary creator gui distributed with ctakes 4.0. It is 99% the same as the version in sandbox. One important difference is that it now produces a database in hsqldb 2.3.4 - compatible with ctakes 4.0.
Another difference (important to you) is that there is an exclusion list in a data file that lists cui / term combinations that can be excluded. By default the medications "toDAY" and "ToMORROW" are in that list. It is obviously done to prevent the frequent false positives that you spoke of in your Amia presentation. For non-vet use these exclusions are pretty valid. Since you are interested in a vet dictionary fi you use the new dictionary creator you can decide whether or not you want them to be included.
Sean
-----Original Message-----
From: James Masanz [mailto:masanz.james@gmail.com]
Sent: Saturday, April 22, 2017 11:32 PM
To: dev@ctakes.apache.org
Subject: Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Dave,
It's an oversight that we didn't document that. Thanks for pointing that out! I'll update the documentation tomorrow and post again with proper details of the command to use.
The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 version of that jar, which is included with the cTAKES 4.0 convenience binary, to open your database. Then you close (shutdown) your DB and it gets converted to 2.3.4 for you. But if the .properties file for your DB indicates it's readonly, you need to have edited the properties file first to remove that. That gives you the idea. I'll write it up properly tomorrow.
Thanks for testing,
-- James
*Sent from my phone.*
On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
I finally had some time this weekend to give 4.0.0rc3 a run. The install and configuration instructions worked perfectly on my Linux laptop and I was able to use CVD to run the clinical pipeline successfully against some of the clinical notes I have here.
However, I ran into a problem when I tried running my own customized pipeline that includes a custom dictionary that I create (to include the SNOMED veterinary extension). I get an exception while it's loading that the dictionary is not the correct version. So, I assume I just need to recreate that custom database? Or is there a migration utility from the old database version to the new version?
I don't remember seeing mention of this, but certainly could have missed it. If it's not in the release notes or install/upgrade instructions it probably should be there.
Thanks to everyone for pulling this release together and getting it out the door!
- Dave
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Posted by James Masanz <ma...@gmail.com>.
Dave,
It's an oversight that we didn't document that. Thanks for pointing that
out! I'll update the documentation tomorrow and post again with proper
details of the command to use.
The short answer is that hsqldb jar includes a GUI. You use the 2.3.4
version of that jar, which is included with the cTAKES 4.0 convenience
binary, to open your database. Then you close (shutdown) your DB and it
gets converted to 2.3.4 for you. But if the .properties file for your DB
indicates it's readonly, you need to have edited the properties file first
to remove that. That gives you the idea. I'll write it up properly tomorrow.
Thanks for testing,
-- James
*Sent from my phone.*
On Apr 22, 2017 9:39 PM, "David Kincaid" <ki...@gmail.com> wrote:
I finally had some time this weekend to give 4.0.0rc3 a run. The install
and configuration instructions worked perfectly on my Linux laptop and I
was able to use CVD to run the clinical pipeline successfully against some
of the clinical notes I have here.
However, I ran into a problem when I tried running my own customized
pipeline that includes a custom dictionary that I create (to include the
SNOMED veterinary extension). I get an exception while it's loading that
the dictionary is not the correct version. So, I assume I just need to
recreate that custom database? Or is there a migration utility from the old
database version to the new version?
I don't remember seeing mention of this, but certainly could have missed
it. If it's not in the release notes or install/upgrade instructions it
probably should be there.
Thanks to everyone for pulling this release together and getting it out the
door!
- Dave