UIMA AS: Client Detects Connection to Broker is Down

[David Wall <da...@us.ibm.com>]

The first pass of my client code creates a new UimaAsynchronousEngine object for
each call to the broker.  Creating the object can take significant time, so I
want to reuse them in an update to the my client code.  Processing the CAS on
the server can take quite a while so the timeout is set to 10 minutes.  The
normal processing time is a minute or less but there are cases that take quite a
bit longer so the timeout is set to account for those cases.

I have questions on error processing.  The scenario of this post is:
   1) Start broker and server components
   2) Start client
   3) Process first CAS -- first time, so create new Engine object
   4) Process next CAS -- reuse Engine object 
   5) Process next CAS -- reuse Engine object
   6) End broker
   7) Process next CAS -- fails at 10 minute mark.
   8) Process next CAS 
   9) Start broker and server components -- CAS supplied in step 8 now runs

What I am looking for is a way for the client to detect the broker is down
before waiting for a timeout (step 7).  Reusing the engine objects appears to be
working well and performance is better.  I'm not sure I can live with the error
detection model, however.  I like the fact that the Engine object automatically
re-connects when the broker comes back on-line (step 9), but I would rather
detect that the broker is down in the client and give my user an error then make
them wait for the timeout.  If I could detect the connection problem then I
could discard the current Engine object and re-create a new one when the next
request comes in.

Any ideas?  Thanks!

Dave

Re: UIMA AS: Client Detects Connection to Broker is Down

[Jaroslaw Cwiklik <ui...@gmail.com>]

Dave, the UIMA AS client is designed to auto recover from broker failures.
Currently there is no override for this. The client will have to change to
support application directive to report an error on JMS problems. The
application could add a new parameter to the Application Context Map (
http://uima.apache.org/d/uima-as-2.3.1/uima_async_scaleout.html#ugr.ref.async.context.map)
which would describe preferred error handling (auto-recover or fail-fast).
I will add a new JIRA to capture this extension to the UIMA AS.

Another option for you to consider is to use a network of brokers with
failover.: http://activemq.apache.org/failover-transport-reference.html.
With that you could use the current UIMA AS code and use ActiveMQs built in
recovery mechanism.

- Jerry


On Thu, Jan 5, 2012 at 9:31 AM, David Wall <da...@us.ibm.com> wrote:

> The first pass of my client code creates a new UimaAsynchronousEngine
> object for
> each call to the broker.  Creating the object can take significant time,
> so I
> want to reuse them in an update to the my client code.  Processing the CAS
> on
> the server can take quite a while so the timeout is set to 10 minutes.  The
> normal processing time is a minute or less but there are cases that take
> quite a
> bit longer so the timeout is set to account for those cases.
>
> I have questions on error processing.  The scenario of this post is:
>   1) Start broker and server components
>   2) Start client
>   3) Process first CAS -- first time, so create new Engine object
>   4) Process next CAS -- reuse Engine object
>   5) Process next CAS -- reuse Engine object
>   6) End broker
>   7) Process next CAS -- fails at 10 minute mark.
>   8) Process next CAS
>   9) Start broker and server components -- CAS supplied in step 8 now runs
>
> What I am looking for is a way for the client to detect the broker is down
> before waiting for a timeout (step 7).  Reusing the engine objects appears
> to be
> working well and performance is better.  I'm not sure I can live with the
> error
> detection model, however.  I like the fact that the Engine object
> automatically
> re-connects when the broker comes back on-line (step 9), but I would rather
> detect that the broker is down in the client and give my user an error
> then make
> them wait for the timeout.  If I could detect the connection problem then I
> could discard the current Engine object and re-create a new one when the
> next
> request comes in.
>
> Any ideas?  Thanks!
>
> Dave
>
>

Re: UIMA AS: Client Detects Connection to Broker is Down

[David Wall <da...@us.ibm.com>]

Jaroslaw Cwiklik <ui...@...> writes:

> 
> Dave, the UIMA AS client is designed to auto recover from broker failures.
> Currently there is no override for this. The client will have to change to
> support application directive to report an error on JMS problems. The
> application could add a new parameter to the Application Context Map (
>
http://uima.apache.org/d/uima-as-2.3.1/uima_async_scaleout.html#ugr.ref.async.context.map)
> which would describe preferred error handling (auto-recover or fail-fast).
> I will add a new JIRA to capture this extension to the UIMA AS.
> 
> Another option for you to consider is to use a network of brokers with
> failover.: http://activemq.apache.org/failover-transport-reference.html.
> With that you could use the current UIMA AS code and use ActiveMQs built in
> recovery mechanism.
> 
> - Jerry
> 

Thanks for the response.  I agree that a redundant set of brokers and workers 
is the ultimate solution.  The servers for a critical application should be
redundant so failures isolated to a specific node are not seen by the client
application.  We will eventually get there.  Until then I'll work with what I
have and knowing the behavior of AS helps me with a solution.  Also thanks for
opening a change request so the client can disable some of the auto-retry
function if that works better for the client.  

Dave