You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by Benjamin Sznajder <be...@il.ibm.com> on 2007/11/11 09:44:00 UTC
Multithreading question
Hi all,
I am interested in using multi-threading in UIMA.
My aim is that the flow runs several annotators in parallel.
One of my annotators is not thread-safe. My question is, then,
Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
that annotators on which this flag is set, are thread-safe?
Regards,
Benjamin.
Re: XmiWriterCasConsumer
Posted by Benjamin Sznajder <be...@il.ibm.com>.
Hi Eddie,
Thanks for your answer!
Benjamin.
"Eddie Epstein"
<eaepstein@gmail.
com> To
uima-user@incubator.apache.org
20/11/2007 19:01 cc
Subject
Please respond to Re: XmiWriterCasConsumer
uima-user@incubat
or.apache.org
Hi Benjamin,
In addition to this one there are several common components in both
packages. There are differences between versions as the components in tools
are intended to support UIMA utilities, like documentAnalyzer and runAE,
while the components in examples are intended to be relatively simple
examples.
Was there something more specific you were looking for?
Eddie
On Nov 20, 2007 10:52 AM, Benjamin Sznajder <be...@il.ibm.com> wrote:
> Hi,
>
> I see that XmiWriterCasConsumer exists in two different jars:
> - uima-tools.jar
> - uima-examples.jar
>
> Is there a difference between both?
>
> Regards,
> Benjamin
>
>
Re: XmiWriterCasConsumer
Posted by Eddie Epstein <ea...@gmail.com>.
Hi Benjamin,
In addition to this one there are several common components in both
packages. There are differences between versions as the components in tools
are intended to support UIMA utilities, like documentAnalyzer and runAE,
while the components in examples are intended to be relatively simple
examples.
Was there something more specific you were looking for?
Eddie
On Nov 20, 2007 10:52 AM, Benjamin Sznajder <be...@il.ibm.com> wrote:
> Hi,
>
> I see that XmiWriterCasConsumer exists in two different jars:
> - uima-tools.jar
> - uima-examples.jar
>
> Is there a difference between both?
>
> Regards,
> Benjamin
>
>
Re: (one other) Multithreading question
Posted by Steve Suppe <ss...@llnl.gov>.
A slightly different (but related question):
I've been playing around with this type of computation. We are loading
data into a DB. We have a small Linux cluster (15 multi-core nodes at the
moment) that we have scaled up to run 224+ instances of our pipeline. I've
noticed for most of our calculations, it's really Oracle that is holding us
back.
In some instances, a 'computation' is classified as one 'medium to long'
data pull from Oracle, a bunch of analysis, and then 'small to large'
insertions of results. I've dabbled in placing static DB connections and
mutexes through the code to guarantee that the instances on a machine only
access the DB one at a time, but are free to run analysis simultaneously
otherwise.
I have also toyed with the idea of locks that allow N number of connections
(instead of only the mutual exclusion one at a time) so that I can
increase the connections to a point, but not overload the system.
Has anyone tried anything like this? Or is anyone else at least running a
similar hardware set-up? It would be great to compare note.
Thanks,
Steve
At 04:09 AM 11/12/2007, Marshall Schor wrote:
>This may not be quite precise enough. Your Annotators will be
>instantiated multiple times, so that a single *instance* of an
>annotator will not be run on multiple threads at once. So - if you have
>non-static fields in your annotator, they do not need to be accessed
>with threading in mind. But if you make use of "static" fields, there is
>only one instance of these, so access to them must be thread-safe.
>
>If your *application* (not your annotator) is multi-threaded, it will
>need to be thread-safe. You can find relevant information about this in
>the tutorial and reference docs for UIMA (search for "thread").
>
>-Marshall
>
>Michael Baessler wrote:
> > Benjamin Sznajder wrote:
> >> Hi all,
> >>
> >> I am interested in using multi-threading in UIMA.
> >> My aim is that the flow runs several annotators in parallel.
> >> One of my annotators is not thread-safe. My question is, then,
> >> Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
> >> that annotators on which this flag is set, are thread-safe?
> >>
> >> Regards,
> >> Benjamin.
> >>
> >>
> > Yes, your annotator have to be thread-safe when you want to run them
> > multi-threaded.
> >
> > -- Michael
> >
> >
XmiWriterCasConsumer
Posted by Benjamin Sznajder <be...@il.ibm.com>.
Hi,
I see that XmiWriterCasConsumer exists in two different jars:
- uima-tools.jar
- uima-examples.jar
Is there a difference between both?
Regards,
Benjamin
Re: Multithreading question
Posted by Marshall Schor <ms...@schor.com>.
This may not be quite precise enough. Your Annotators will be
instantiated multiple times, so that a single *instance* of an
annotator will not be run on multiple threads at once. So - if you have
non-static fields in your annotator, they do not need to be accessed
with threading in mind. But if you make use of "static" fields, there is
only one instance of these, so access to them must be thread-safe.
If your *application* (not your annotator) is multi-threaded, it will
need to be thread-safe. You can find relevant information about this in
the tutorial and reference docs for UIMA (search for "thread").
-Marshall
Michael Baessler wrote:
> Benjamin Sznajder wrote:
>> Hi all,
>>
>> I am interested in using multi-threading in UIMA.
>> My aim is that the flow runs several annotators in parallel.
>> One of my annotators is not thread-safe. My question is, then,
>> Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
>> that annotators on which this flag is set, are thread-safe?
>>
>> Regards,
>> Benjamin.
>>
>>
> Yes, your annotator have to be thread-safe when you want to run them
> multi-threaded.
>
> -- Michael
>
>
Re: Multithreading question
Posted by Michael Baessler <mb...@michael-baessler.de>.
Benjamin Sznajder wrote:
> Hi all,
>
> I am interested in using multi-threading in UIMA.
> My aim is that the flow runs several annotators in parallel.
> One of my annotators is not thread-safe. My question is, then,
> Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
> that annotators on which this flag is set, are thread-safe?
>
> Regards,
> Benjamin.
>
>
Yes, your annotator have to be thread-safe when you want to run them
multi-threaded.
-- Michael