You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by "Miller, Timothy" <Ti...@childrens.harvard.edu> on 2019/01/18 18:29:26 UTC
Re: uima-as examples [EXTERNAL]
Greg - I've developed a cluster-like architecture that uses Docker-wrapped UIMA-AS components on AWS for scalability. It's a work in progress but it might be helpful:
https://github.com/tmills/ctakes-docker
Tim
-----Original Message-----
From: Greg Silverman <gms@umn.edu<mailto:Greg%20Silverman%20%3cgms@umn.edu%3e>>
Reply-to: <de...@ctakes.apache.org>
To: dev@ctakes.apache.org<ma...@ctakes.apache.org>
Cc: Raymond Finzel <finze006@umn.edu<mailto:Raymond%20Finzel%20%3cfinze006@umn.edu%3e>>, Reed McEwan <rmcewan@umn.edu<mailto:Reed%20McEwan%20%3crmcewan@umn.edu%3e>>
Subject: Re: uima-as examples [EXTERNAL]
Date: Fri, 18 Jan 2019 12:23:53 -0600
Thanks Peter,
The architecture for our project (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_nlpie_nlp-2Dadapt-2Dkube&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=fvf05Pvhnq2FEnKxgYHuXibuP5Is9-bZCEE8-cbqq8M&e=,
uima-as branch under current development), relies heavily on uima-as to
work in conjunction with ActiveMQ and a home spun multiplexer/collection
processing client to do all the heavy lifting for the nlp-engines we're
using. Currently, CLAMP, and BioMedICUS both support UIMA-AS out-of-the-box
(I'm looking into MetaMap, as I type this).
To the best of my knowledge, the MQ and broker work together (at least in
ActiveMQ).
Given the volume of documents we need to process and the constraint of
being tied to UIMA, UIMA-AS is the easiest option for implementing at
scale, for both speed and fault tolerance.
If anyone has done any work trying to integrate UIMA-AS into cTAKES we
would be very interested in this. Retrofitting a different solution into
our architecture at this time is not feasible.
Thanks very much!
Best!
Greg-
On Thu, Jan 17, 2019 at 10:08 PM Peter Abramowitsch <pa...@gmail.com>>
wrote:
I used a completely different approach that allows parallel but not async
processing. Multiple [analysis engine+cas] pair objects pre-instantiated
into into a threadsafe pool running behind a web service interface. We can
fully saturate a single ctakes server process using multiple client
processes talking to that API each working synchronously and arriving at an
overall speed of 10-15 6K notes per second on a single server process.
I haven't used AS but it looks as if that middleware could have too many
moving parts for our needs. They would generate many wakeups and context
switches adding undesired latency as a request makes its way to the
server. I'm assuming that in AS, the broker and the MQ are separate
processes and not just in-process subsystems to the ctakes server process.
Is that right?
On Thu, Jan 17, 2019 at 4:09 PM Greg Silverman <gm...@umn.edu>> wrote:
Anyone out there developed a pipeline using UIMA-AS, as opposed to the
CPE/CPM file reader?
Thanks in advance!
Greg--
--
Greg M. Silverman
Senior Systems Developer
NLP/IE <https://urldefense.proofpoint.com/v2/url?u=https-3A__healthinformatics.umn.edu_research_nlpie-2Dgroup&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=M9wwarnfKqExquzT7isXdu4-A3-sDXx0Rar1-Y1HrlA&e=>
Cardiovascular Informatics <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.med.umn.edu_cardiology_&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=FdX-_52HBO3FixbIlX0HB8sdIvmjsTvd-lCM_PL3qII&e=>
University of Minnesota
gms@umn.edu<ma...@umn.edu>
› evaluate-it.org ‹
Re: uima-as examples [EXTERNAL]
Posted by Greg Silverman <gm...@umn.edu>.
Tim,
This is perfect!
Very much appreciated!
Greg--
On Fri, Jan 18, 2019 at 12:36 PM Miller, Timothy <
Timothy.Miller@childrens.harvard.edu> wrote:
> Greg - I've developed a cluster-like architecture that uses Docker-wrapped
> UIMA-AS components on AWS for scalability. It's a work in progress but it
> might be helpful:
> https://github.com/tmills/ctakes-docker
> Tim
>
>
> -----Original Message-----
> From: Greg Silverman <gms@umn.edu<mailto:Greg%20Silverman%20%3cgms@umn.edu
> %3e>>
> Reply-to: <de...@ctakes.apache.org>
> To: dev@ctakes.apache.org<ma...@ctakes.apache.org>
> Cc: Raymond Finzel <finze006@umn.edu<mailto:
> Raymond%20Finzel%20%3cfinze006@umn.edu%3e>>, Reed McEwan <rmcewan@umn.edu
> <mailto:Reed%20McEwan%20%3crmcewan@umn.edu%3e>>
> Subject: Re: uima-as examples [EXTERNAL]
> Date: Fri, 18 Jan 2019 12:23:53 -0600
>
>
> Thanks Peter,
> The architecture for our project (
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_nlpie_nlp-2Dadapt-2Dkube&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=fvf05Pvhnq2FEnKxgYHuXibuP5Is9-bZCEE8-cbqq8M&e=
> ,
> uima-as branch under current development), relies heavily on uima-as to
> work in conjunction with ActiveMQ and a home spun multiplexer/collection
> processing client to do all the heavy lifting for the nlp-engines we're
> using. Currently, CLAMP, and BioMedICUS both support UIMA-AS out-of-the-box
> (I'm looking into MetaMap, as I type this).
>
> To the best of my knowledge, the MQ and broker work together (at least in
> ActiveMQ).
>
> Given the volume of documents we need to process and the constraint of
> being tied to UIMA, UIMA-AS is the easiest option for implementing at
> scale, for both speed and fault tolerance.
>
> If anyone has done any work trying to integrate UIMA-AS into cTAKES we
> would be very interested in this. Retrofitting a different solution into
> our architecture at this time is not feasible.
>
> Thanks very much!
>
> Best!
>
> Greg-
>
>
>
> On Thu, Jan 17, 2019 at 10:08 PM Peter Abramowitsch <
> pabramowitsch@gmail.com<ma...@gmail.com>>
> wrote:
>
>
>
> I used a completely different approach that allows parallel but not async
> processing. Multiple [analysis engine+cas] pair objects pre-instantiated
> into into a threadsafe pool running behind a web service interface. We can
> fully saturate a single ctakes server process using multiple client
> processes talking to that API each working synchronously and arriving at an
> overall speed of 10-15 6K notes per second on a single server process.
>
> I haven't used AS but it looks as if that middleware could have too many
> moving parts for our needs. They would generate many wakeups and context
> switches adding undesired latency as a request makes its way to the
> server. I'm assuming that in AS, the broker and the MQ are separate
> processes and not just in-process subsystems to the ctakes server process.
> Is that right?
>
> On Thu, Jan 17, 2019 at 4:09 PM Greg Silverman <gms@umn.edu<mailto:
> gms@umn.edu>> wrote:
>
>
>
> Anyone out there developed a pipeline using UIMA-AS, as opposed to the
> CPE/CPM file reader?
>
> Thanks in advance!
>
> Greg--
>
> --
> Greg M. Silverman
> Senior Systems Developer
> NLP/IE <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__healthinformatics.umn.edu_research_nlpie-2Dgroup&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=M9wwarnfKqExquzT7isXdu4-A3-sDXx0Rar1-Y1HrlA&e=
> >
> Cardiovascular Informatics <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.med.umn.edu_cardiology_&d=DwIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=WEY8xYYIUiTWnZDnwU72eUiyHXNWFAi3vY9DMayfV-g&s=FdX-_52HBO3FixbIlX0HB8sdIvmjsTvd-lCM_PL3qII&e=
> >
> University of Minnesota
> gms@umn.edu<ma...@umn.edu>
>
> › evaluate-it.org ‹
>
>
>
>
>
>
>
>
>
>
--
Greg M. Silverman
Senior Systems Developer
NLP/IE <https://healthinformatics.umn.edu/research/nlpie-group>
Cardiovascular Informatics <http://www.med.umn.edu/cardiology/>
University of Minnesota
gms@umn.edu
› evaluate-it.org ‹