Re: (one other) Multithreading question

[Steve Suppe <ss...@llnl.gov>]

A slightly different (but related question):

I've been playing around with this type of computation.  We are loading 
data into a DB.  We have a small Linux cluster (15 multi-core nodes at the 
moment) that we have scaled up to run 224+ instances of our pipeline.  I've 
noticed for most of our calculations, it's really Oracle that is holding us 
back.

In some instances, a 'computation' is classified as one 'medium to long' 
data pull from Oracle, a bunch of analysis, and then 'small to large' 
insertions of results.  I've dabbled in placing static DB connections and 
mutexes through the code to guarantee that the instances on a machine only 
access the DB one at a time, but are free to run analysis simultaneously 
otherwise.

I have also toyed with the idea of locks that allow N number of connections 
(instead of only  the mutual exclusion one at a time) so that I can 
increase the connections to a point, but not overload the system.

Has anyone tried anything like this?  Or is anyone else at least running a 
similar hardware set-up?  It would be great to compare note.

Thanks,
Steve

At 04:09 AM 11/12/2007, Marshall Schor wrote:
>This may not be quite precise enough.  Your Annotators will be
>instantiated multiple times,  so that a single *instance* of an
>annotator will not be run on multiple threads at once.  So - if you have
>non-static fields in your annotator, they do not need to be accessed
>with threading in mind. But if you make use of "static" fields, there is
>only one instance of these, so access to them must be thread-safe.
>
>If your *application*  (not your annotator) is multi-threaded, it will
>need to be thread-safe. You can find relevant information about this in
>the tutorial and reference docs for UIMA (search for "thread").
>
>-Marshall
>
>Michael Baessler wrote:
> > Benjamin Sznajder wrote:
> >> Hi all,
> >>
> >> I am interested in using multi-threading in UIMA.
> >> My aim is that the flow runs several annotators in parallel.
> >> One of my annotators is not thread-safe. My question is, then,
> >> Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
> >> that annotators on which this flag is set, are thread-safe?
> >>
> >> Regards,
> >> Benjamin.
> >>
> >>
> > Yes, your annotator have to be thread-safe when you want to run them
> > multi-threaded.
> >
> > -- Michael
> >
> >