You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by Charles Bearden <Ch...@uth.tmc.edu> on 2011/08/16 17:48:04 UTC
Performance of CPE/CPM vs AS
Thank you Jerry & Eddie for your responses to my previous questions. I
appreciate the opportunity to learn.
Based on a little testing, I'm starting to think that AS is not designed for
performance-enhancing scale-out, but maybe rather for architectural clarity. I
have a CPE that has a collection reader that reads sentences from a database,
and an aggregate AE that is the cTAKES AggregatePlaintextProcessor (using our
dictionary for dictionary lookup) plus an AE that writes the concept annotations
to a database. When I put these together as a CPE and run it against a test set
of 2553 sentences, it takes about one minute, sometimes a few seconds less. The
CpmFrame GUI indicates that the CR accounts for about 5% of the processing time,
and the AE for the other 95%, with the LVG annotator & dictionary lookup each
accounting for between 35%-45%.
When I use the same CR & aggregate AE like this:
runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
-c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
-d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
it takes 16 minutes to process the same 2553 sentences.
Deploy_DictionaryTest.xml is the deployment descriptor; you can see its contents
here: <http://pastebin.com/6nhuaC4H>.
When I deploy the AE five times with 'deployAsyncService.sh' like this:
deployAsyncService.sh \
sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
and then use 'runRemoteAsyncAE.sh' to connect the CR to the input queue like this:
runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
-c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml
it still takes 12 minutes to process the 2553 sentences. I can see from the log
files that the processing is being scaled out. Given that in the CPE the CASes
spent only 5% of their time in the CR, I'm skeptical it has become the
bottleneck, though I could be wrong. I'm just wondering if this kind of
performance difference is expected.
Thanks,
Chuck
--
Chuck Bearden
Programmer Analyst IV
The University of Texas Health Science Center at Houston
School of Biomedical Informatics
Email: Charles.F.Bearden@uth.tmc.edu
Phone: 713.500.9672
Re: Performance of CPE/CPM vs AS
Posted by Charles Bearden <Ch...@uth.tmc.edu>.
Hi Marshall,
Thanks for your reply. I should have included details of our machine. It has
four CPUs each with 8 cores; here's the /proc/cpuinfo stanza for one of the
cores: <http://pastebin.com/F8YXs7XL>
The machine has 64G RAM; I was running dstat at the same time as the AS
pipeline, and it indicated no paging. Here's a typical snapshot of the memory
part of top during the run:
Mem: 66096700k total, 43877612k used, 22219088k free, 608092k buffers
Swap: 47321080k total, 5097644k used, 42223436k free, 22594808k cached
I'll intersperse further comments in your text.
On 08/16/2011 11:25 AM, Marshall Schor wrote:
> When looking at performance, it's important to get more details around the
> possibilities. For instance, is the machine your running on a multi-core
> machine (meaning it can run multiple threads at the same time, for increased
> performance), and if so, how many cores? The Intel i7 I think supports 8
> threads at once, for instance.
>
> If you have a machine that is capable of this kind of performance, then you can
> configure UIMA to take advantage of this, (or not). For instance, in UIMA-AS
> you could specify limiting the number of things a pipeline could be doing at
> once, to just one thing (so, for example, if you had 8 cores, but wanted the
> pipeline to just use one, so the other cores could be used for other things, you
> can specify that.
>
> You actually did that, in your deployment descriptor, by limiting the scaleout
> to 1 and the size of the CAS pool to 1. So I would not expect any "scaleout"
> effect from this kind of configuration :-).
Right. In this case, I was trying to compare apples to applies, i.e. a
single-threaded CPE with a single-threaded AS pipeline. The AS pipeline was 16x
slower than the CPE. Even an AS pipeline with 5 deployments of of the AE was 12x
slower than the single-threaded CPE. Please note that I'm definitely open to the
possibility that I'm overlooking something really elementary, and I'm willing to
be shown the error of my ways.
> Likewise, in the CPE you can specify the scaleout of the AE's, to take advantage
> of multiple cores. You didn't say what you specified here, so I don't know...
Unfortunately, some components of the AE I'm using are not thread-safe, so I
can't take advantage of scaleout within a single AE instance: I deploy each
instance separately so that it has its own JVM. This script shows how I deployed
the five instances with one CR: <http://pastebin.com/sczHwRuD>
> Also, CPEs can be run in "Integrated" or other kinds of modes. In Integrated,
> this means that the CPE runs everything on one machine, in the same JVM. So
> there is no serialization (sending / receiving the CAS over a TCP/IP connection
> to/from a remote machine). Of course, that limits the scaling you get, to
> whatever the 1 machine can supply. Again, you didn't specify what kind of CPE
> deployment you did.
Here is the CPE desriptor I used: <http://pastebin.com/ipKyCc8B> Of course, many
of the parameters are hidden in the CR and AE descriptors. Not sure if they have
much bearing on what I'm seeing.
Thanks, and again, I really appreciate the comments from the members of this list.
Chuck
> (Note that you can configure UIMA-AS to also run in one JVM with no
> serialization, but I don't think that was the case here.)
>
> Finally, if you are running in your test in the UIMA-AS case, the client code,
> the broker, and the service all on one machine, is that machine "paging"? This
> may be dependent on the size of your CASes, etc.
>
> -Marshall
>
>
> On 8/16/2011 11:48 AM, Charles Bearden wrote:
>> Thank you Jerry& Eddie for your responses to my previous questions. I
>> appreciate the opportunity to learn.
>>
>> Based on a little testing, I'm starting to think that AS is not designed for
>> performance-enhancing scale-out, but maybe rather for architectural clarity. I
>> have a CPE that has a collection reader that reads sentences from a database,
>> and an aggregate AE that is the cTAKES AggregatePlaintextProcessor (using our
>> dictionary for dictionary lookup) plus an AE that writes the concept
>> annotations to a database. When I put these together as a CPE and run it
>> against a test set of 2553 sentences, it takes about one minute, sometimes a
>> few seconds less. The CpmFrame GUI indicates that the CR accounts for about 5%
>> of the processing time, and the AE for the other 95%, with the LVG annotator&
>> dictionary lookup each accounting for between 35%-45%.
>>
>> When I use the same CR& aggregate AE like this:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
>> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>
>> it takes 16 minutes to process the same 2553 sentences.
>> Deploy_DictionaryTest.xml is the deployment descriptor; you can see its
>> contents here:<http://pastebin.com/6nhuaC4H>.
>>
>> When I deploy the AE five times with 'deployAsyncService.sh' like this:
>>
>> deployAsyncService.sh \
>> sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>
>> and then use 'runRemoteAsyncAE.sh' to connect the CR to the input queue like
>> this:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml
>>
>> it still takes 12 minutes to process the 2553 sentences. I can see from the
>> log files that the processing is being scaled out. Given that in the CPE the
>> CASes spent only 5% of their time in the CR, I'm skeptical it has become the
>> bottleneck, though I could be wrong. I'm just wondering if this kind of
>> performance difference is expected.
>>
>> Thanks,
>> Chuck
Re: Performance of CPE/CPM vs AS
Posted by Marshall Schor <ms...@schor.com>.
When looking at performance, it's important to get more details around the
possibilities. For instance, is the machine your running on a multi-core
machine (meaning it can run multiple threads at the same time, for increased
performance), and if so, how many cores? The Intel i7 I think supports 8
threads at once, for instance.
If you have a machine that is capable of this kind of performance, then you can
configure UIMA to take advantage of this, (or not). For instance, in UIMA-AS
you could specify limiting the number of things a pipeline could be doing at
once, to just one thing (so, for example, if you had 8 cores, but wanted the
pipeline to just use one, so the other cores could be used for other things, you
can specify that.
You actually did that, in your deployment descriptor, by limiting the scaleout
to 1 and the size of the CAS pool to 1. So I would not expect any "scaleout"
effect from this kind of configuration :-).
Likewise, in the CPE you can specify the scaleout of the AE's, to take advantage
of multiple cores. You didn't say what you specified here, so I don't know...
Also, CPEs can be run in "Integrated" or other kinds of modes. In Integrated,
this means that the CPE runs everything on one machine, in the same JVM. So
there is no serialization (sending / receiving the CAS over a TCP/IP connection
to/from a remote machine). Of course, that limits the scaling you get, to
whatever the 1 machine can supply. Again, you didn't specify what kind of CPE
deployment you did.
(Note that you can configure UIMA-AS to also run in one JVM with no
serialization, but I don't think that was the case here.)
Finally, if you are running in your test in the UIMA-AS case, the client code,
the broker, and the service all on one machine, is that machine "paging"? This
may be dependent on the size of your CASes, etc.
-Marshall
On 8/16/2011 11:48 AM, Charles Bearden wrote:
> Thank you Jerry & Eddie for your responses to my previous questions. I
> appreciate the opportunity to learn.
>
> Based on a little testing, I'm starting to think that AS is not designed for
> performance-enhancing scale-out, but maybe rather for architectural clarity. I
> have a CPE that has a collection reader that reads sentences from a database,
> and an aggregate AE that is the cTAKES AggregatePlaintextProcessor (using our
> dictionary for dictionary lookup) plus an AE that writes the concept
> annotations to a database. When I put these together as a CPE and run it
> against a test set of 2553 sentences, it takes about one minute, sometimes a
> few seconds less. The CpmFrame GUI indicates that the CR accounts for about 5%
> of the processing time, and the AE for the other 95%, with the LVG annotator &
> dictionary lookup each accounting for between 35%-45%.
>
> When I use the same CR & aggregate AE like this:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>
> it takes 16 minutes to process the same 2553 sentences.
> Deploy_DictionaryTest.xml is the deployment descriptor; you can see its
> contents here: <http://pastebin.com/6nhuaC4H>.
>
> When I deploy the AE five times with 'deployAsyncService.sh' like this:
>
> deployAsyncService.sh \
> sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>
> and then use 'runRemoteAsyncAE.sh' to connect the CR to the input queue like
> this:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml
>
> it still takes 12 minutes to process the 2553 sentences. I can see from the
> log files that the processing is being scaled out. Given that in the CPE the
> CASes spent only 5% of their time in the CR, I'm skeptical it has become the
> bottleneck, though I could be wrong. I'm just wondering if this kind of
> performance difference is expected.
>
> Thanks,
> Chuck
Performance of CPE/CPM vs AS
Posted by Eddie Epstein <ea...@gmail.com>.
Accidentally replied off list ...
---------- Forwarded message ----------
From: Eddie Epstein <ea...@gmail.com>
Date: Tue, Aug 16, 2011 at 2:45 PM
Subject: Re: Performance of CPE/CPM vs AS
To: Charles Bearden <Ch...@uth.tmc.edu>
> Thanks again for your reply. I thought that I was deploying the pipeline in
> one AS process with the first option for running it:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>
> It looks like one process in the output of ps. I'm just surprised that the
> performance is so much slower (16x slower).
Right, all in one process, but the connection between client and
service is the same used between multiple processes. As a quick test,
create a new aggregate with these two delegates:
SentencesFromDBReader.xml and
SbmiUmlsSmallAggregatePlaintextProcessor.xml. Then create a deployment
descriptor for this aggregate, say
Deploy_OneProcessDictionaryTest.xml, and test it with:
runRemoteAsyncAE.sh tcp://localhost:61616 OneProcessQueue \
-d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_OneProcessDictionaryTest.xml
Without a collection reader runRemoteAsyncAE will send a single empty
CAS to the service. This will kick off the embedded collection reader
in the aggregate, and hopefully you'll see times similar to the CPE.
> To create a pipeline with an architecture like Figure 5, I would use the
> example in "4.6. Asynchronous Client API Usage Scenarios" on p. 30 of the
> uima_async_scaleout.pdf for 2.3.1?
That would be one way. The important points are 1) to send a CAS which
points at some subset of the collection, and 2) change the embedded
collection reader inside the service to a CasMultiplier which can
access that CAS and generate the sub-collection of CASes to the
pipeline. Given these 2, a static set of CASes to be sent to the
service could be created and runRemoteAsyncAE used to send them.
Eddie
Re: Performance of CPE/CPM vs AS
Posted by Eddie Epstein <ea...@gmail.com>.
> I get it now. When I put the CR into the aggregate AE and ran
> 'runRemoteAsyncAE.sh' without the '-c' flag, it was within a few seconds of
> being as fast as the CPE.
That's good :)
The few seconds are because the CPE/CPM runs the CR in a separate
thread than the AE, but the simple UIMA AS deployment is running then
in the same thread. Should be closer if the aggregate is deployed in
the "async" mode. Also, if AE is thread-safe, try increasing its
number of instances.
> Thanks for the pointers. One take-away for me seems to be that UIMA AS might
> not be a means to scale for performance if you have to run service instances
> remotely.
Running UIMA AS services remotely can provide almost linear scale out
performance, as long as data flow is designed carefully.
Eddie
On Tue, Aug 16, 2011 at 4:43 PM, Charles Bearden
<Ch...@uth.tmc.edu> wrote:
> On 08/16/2011 01:45 PM, Eddie Epstein wrote:
>>>
>>> Thanks again for your reply. I thought that I was deploying the pipeline
>>> in
>>> one AS process with the first option for running it:
>>>
>>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
>>> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>>
>>> It looks like one process in the output of ps. I'm just surprised that
>>> the
>>> performance is so much slower (16x slower).
>>
>> Right, all in one process, but the connection between client and
>> service is the same used between multiple processes. As a quick test,
>> create a new aggregate with these two delegates:
>> SentencesFromDBReader.xml and
>> SbmiUmlsSmallAggregatePlaintextProcessor.xml. Then create a deployment
>> descriptor for this aggregate, say
>> Deploy_OneProcessDictionaryTest.xml, and test it with:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 OneProcessQueue \
>> -d
>> sbmi-ctsa/desc/asynchronous_scaleout/Deploy_OneProcessDictionaryTest.xml
>>
>> Without a collection reader runRemoteAsyncAE will send a single empty
>> CAS to the service. This will kick off the embedded collection reader
>> in the aggregate, and hopefully you'll see times similar to the CPE.
>
> I get it now. When I put the CR into the aggregate AE and ran
> 'runRemoteAsyncAE.sh' without the '-c' flag, it was within a few seconds of
> being as fast as the CPE.
>
> Thanks for the pointers. One take-away for me seems to be that UIMA AS might
> not be a means to scale for performance if you have to run service instances
> remotely. What I've been doing is to run a bunch of CPEs in parallel, using
> the modulo operator in the SQL of the CR to ensure that each CR is pulling
> data from its own partition of the collection, e.g.
>
> SELECT TEXT
> FROM DOCUMENTS
> WHERE ID % 25 = x
>
> where each of the 25 instances will have a different number from [0, 1, 2, 3
> …] for its 'x'.
>
> Thanks again to all who responded. I've learned a lot.
>
> Chuck
>
>>> To create a pipeline with an architecture like Figure 5, I would use the
>>> example in "4.6. Asynchronous Client API Usage Scenarios" on p. 30 of the
>>> uima_async_scaleout.pdf for 2.3.1?
>>
>> That would be one way. The important points are 1) to send a CAS which
>> points at some subset of the collection, and 2) change the embedded
>> collection reader inside the service to a CasMultiplier which can
>> access that CAS and generate the sub-collection of CASes to the
>> pipeline. Given these 2, a static set of CASes to be sent to the
>> service could be created and runRemoteAsyncAE used to send them.
>>
>> Eddie
>
>
Re: Performance of CPE/CPM vs AS
Posted by Charles Bearden <Ch...@uth.tmc.edu>.
On 08/16/2011 01:45 PM, Eddie Epstein wrote:
>> Thanks again for your reply. I thought that I was deploying the pipeline in
>> one AS process with the first option for running it:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
>> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>
>> It looks like one process in the output of ps. I'm just surprised that the
>> performance is so much slower (16x slower).
>
> Right, all in one process, but the connection between client and
> service is the same used between multiple processes. As a quick test,
> create a new aggregate with these two delegates:
> SentencesFromDBReader.xml and
> SbmiUmlsSmallAggregatePlaintextProcessor.xml. Then create a deployment
> descriptor for this aggregate, say
> Deploy_OneProcessDictionaryTest.xml, and test it with:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 OneProcessQueue \
> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_OneProcessDictionaryTest.xml
>
> Without a collection reader runRemoteAsyncAE will send a single empty
> CAS to the service. This will kick off the embedded collection reader
> in the aggregate, and hopefully you'll see times similar to the CPE.
I get it now. When I put the CR into the aggregate AE and ran
'runRemoteAsyncAE.sh' without the '-c' flag, it was within a few seconds of
being as fast as the CPE.
Thanks for the pointers. One take-away for me seems to be that UIMA AS might not
be a means to scale for performance if you have to run service instances
remotely. What I've been doing is to run a bunch of CPEs in parallel, using the
modulo operator in the SQL of the CR to ensure that each CR is pulling data from
its own partition of the collection, e.g.
SELECT TEXT
FROM DOCUMENTS
WHERE ID % 25 = x
where each of the 25 instances will have a different number from [0, 1, 2, 3 …]
for its 'x'.
Thanks again to all who responded. I've learned a lot.
Chuck
>> To create a pipeline with an architecture like Figure 5, I would use the
>> example in "4.6. Asynchronous Client API Usage Scenarios" on p. 30 of the
>> uima_async_scaleout.pdf for 2.3.1?
>
> That would be one way. The important points are 1) to send a CAS which
> points at some subset of the collection, and 2) change the embedded
> collection reader inside the service to a CasMultiplier which can
> access that CAS and generate the sub-collection of CASes to the
> pipeline. Given these 2, a static set of CASes to be sent to the
> service could be created and runRemoteAsyncAE used to send them.
>
> Eddie
Re: Performance of CPE/CPM vs AS
Posted by Charles Bearden <Ch...@uth.tmc.edu>.
On 08/16/2011 11:15 AM, Eddie Epstein wrote:
> The CPE runs everything in the same process. UIMA AS could deploy this
> pipeline in one process and get the same performance as the CPE.
Thanks again for your reply. I thought that I was deploying the pipeline in one
AS process with the first option for running it:
runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
-c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
-d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
It looks like one process in the output of ps. I'm just surprised that the
performance is so much slower (16x slower).
> Scaling out to multiple processes incurs overhead, which for UIMA AS
> essentially consists of CAS serialization and communication. Figure 5
> on http://uima.apache.org/doc-uimaas-what.html will have much lower
> overhead for this scenario.
To create a pipeline with an architecture like Figure 5, I would use the example
in "4.6. Asynchronous Client API Usage Scenarios" on p. 30 of the
uima_async_scaleout.pdf for 2.3.1?
Thanks,
Chuck
> Eddie
>
> On Tue, Aug 16, 2011 at 11:48 AM, Charles Bearden
> <Ch...@uth.tmc.edu> wrote:
>> Thank you Jerry& Eddie for your responses to my previous questions. I
>> appreciate the opportunity to learn.
>>
>> Based on a little testing, I'm starting to think that AS is not designed for
>> performance-enhancing scale-out, but maybe rather for architectural clarity.
>> I have a CPE that has a collection reader that reads sentences from a
>> database, and an aggregate AE that is the cTAKES AggregatePlaintextProcessor
>> (using our dictionary for dictionary lookup) plus an AE that writes the
>> concept annotations to a database. When I put these together as a CPE and
>> run it against a test set of 2553 sentences, it takes about one minute,
>> sometimes a few seconds less. The CpmFrame GUI indicates that the CR
>> accounts for about 5% of the processing time, and the AE for the other 95%,
>> with the LVG annotator& dictionary lookup each accounting for between
>> 35%-45%.
>>
>> When I use the same CR& aggregate AE like this:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
>> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>
>> it takes 16 minutes to process the same 2553 sentences.
>> Deploy_DictionaryTest.xml is the deployment descriptor; you can see its
>> contents here:<http://pastebin.com/6nhuaC4H>.
>>
>> When I deploy the AE five times with 'deployAsyncService.sh' like this:
>>
>> deployAsyncService.sh \
>> sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>>
>> and then use 'runRemoteAsyncAE.sh' to connect the CR to the input queue like
>> this:
>>
>> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
>> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml
>>
>> it still takes 12 minutes to process the 2553 sentences. I can see from the
>> log files that the processing is being scaled out. Given that in the CPE the
>> CASes spent only 5% of their time in the CR, I'm skeptical it has become the
>> bottleneck, though I could be wrong. I'm just wondering if this kind of
>> performance difference is expected.
>>
>> Thanks,
>> Chuck
>> --
>> Chuck Bearden
>> Programmer Analyst IV
>> The University of Texas Health Science Center at Houston
>> School of Biomedical Informatics
>> Email: Charles.F.Bearden@uth.tmc.edu
>> Phone: 713.500.9672
Re: Performance of CPE/CPM vs AS
Posted by Eddie Epstein <ea...@gmail.com>.
The CPE runs everything in the same process. UIMA AS could deploy this
pipeline in one process and get the same performance as the CPE.
Scaling out to multiple processes incurs overhead, which for UIMA AS
essentially consists of CAS serialization and communication. Figure 5
on http://uima.apache.org/doc-uimaas-what.html will have much lower
overhead for this scenario.
Eddie
On Tue, Aug 16, 2011 at 11:48 AM, Charles Bearden
<Ch...@uth.tmc.edu> wrote:
> Thank you Jerry & Eddie for your responses to my previous questions. I
> appreciate the opportunity to learn.
>
> Based on a little testing, I'm starting to think that AS is not designed for
> performance-enhancing scale-out, but maybe rather for architectural clarity.
> I have a CPE that has a collection reader that reads sentences from a
> database, and an aggregate AE that is the cTAKES AggregatePlaintextProcessor
> (using our dictionary for dictionary lookup) plus an AE that writes the
> concept annotations to a database. When I put these together as a CPE and
> run it against a test set of 2553 sentences, it takes about one minute,
> sometimes a few seconds less. The CpmFrame GUI indicates that the CR
> accounts for about 5% of the processing time, and the AE for the other 95%,
> with the LVG annotator & dictionary lookup each accounting for between
> 35%-45%.
>
> When I use the same CR & aggregate AE like this:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml \
> -d sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>
> it takes 16 minutes to process the same 2553 sentences.
> Deploy_DictionaryTest.xml is the deployment descriptor; you can see its
> contents here: <http://pastebin.com/6nhuaC4H>.
>
> When I deploy the AE five times with 'deployAsyncService.sh' like this:
>
> deployAsyncService.sh \
> sbmi-ctsa/desc/asynchronous_scaleout/Deploy_DictionaryTest.xml
>
> and then use 'runRemoteAsyncAE.sh' to connect the CR to the input queue like
> this:
>
> runRemoteAsyncAE.sh tcp://localhost:61616 CollectionReader \
> -c sbmi-ctsa/desc/asynchronous_scaleout/SentencesFromDBReader.xml
>
> it still takes 12 minutes to process the 2553 sentences. I can see from the
> log files that the processing is being scaled out. Given that in the CPE the
> CASes spent only 5% of their time in the CR, I'm skeptical it has become the
> bottleneck, though I could be wrong. I'm just wondering if this kind of
> performance difference is expected.
>
> Thanks,
> Chuck
> --
> Chuck Bearden
> Programmer Analyst IV
> The University of Texas Health Science Center at Houston
> School of Biomedical Informatics
> Email: Charles.F.Bearden@uth.tmc.edu
> Phone: 713.500.9672
>
>
>