You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by nelson rivera <ne...@gmail.com> on 2016/09/21 19:42:15 UTC
Broken conections after ACTIVEMQ restart
When start ActiveMQ and deploys an annotator as service, processing is
executed correctly, but when the broker is stopped and subsequently
restarted. UIMA-AS log show the lines below and after this the api
client uima-as is not notified in the listener when a cas is
processed, any more.
can help me?:
02:08:25.570 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
02:08:25.570 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
02:08:25.570 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
02:09:02.222 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
Established Connection to Broker: tcp://localhost:61616
02:09:02.317 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
02:09:02.317 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
org.apache.activemq.ConnectionClosedException: The connection is
already closed02:09:02.356 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
02:09:02.357 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
javax.jms.IllegalStateException: The Consumer is closed.
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
My service is a aggregate analysis engine that receives input CAS
with the direcction of a directory of documents, the first component
is a MultiplerCas that for each document generate a child CAS, each
child CAS continues the flow that are other primitive annotators
(primitive annotators in c++ that can throw exceptions), then exists
other MultiplerCas that acts as Merger, waiting the complete
processing of each child CAS and finally generate a child CAS where
each child CAS previously processed is a view of CAS, this a child CAS
that is composed of view continúa el flujo a otro AE, que espera un
CAS de esta manera.
I will send you my uima-as descriptor, to understand better my
service. All delegates are co-located.
i will trys with uima-as-2.9.0 snapshot how you say, to see how it behaves
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
You had reason, i had problems with older versions installed. With
this snapshot already the queues in service cas multiplier are
recovered. I can consider this theme closed. Thank you for making high
quality software.
2016-11-01 13:37 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Nelson, using the same snapshot I see a different behavior of UIMA-AS
> service when a broker is bounced:
>
> 12:21:35.452 - 27:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
> WARNING: Service: Test Aggregate TAE Runtime Exception
> 12:21:35.452 - 27:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
> WARNING: Jms Listener Failed. Endpoint:
> temp-queue://ID:bluejws65-33772-1478017289483-1:1\
> :1 Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 12:21:35.453 - 36:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
> WARNING: Service: Test Aggregate TAE Runtime Exception
> 12:21:35.453 - 36:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
> WARNING: Jms Listener Failed. Endpoint: SecondLevelTaeQueue Managed By:
> tcp://localhost:61616 Reason\
> : javax.jms.JMSException: java.io.EOFException
> 12:21:35.453 - 27:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING: Uima AS Service:Test Aggregate TAE Listener Unable To Connect To
> Broker: tcp\
> ://localhost:61616 Retrying Until Successful ...
> 12:21:35.453 - 36:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING: Uima AS Service:Test Aggregate TAE Listener Unable To Connect To
> Broker: tcp\
> ://localhost:61616 Retrying Until Successful ...
> 12:21:35.454 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer$3.destroy:
> WARNING:
> org.apache.activemq.ConnectionFailedException: The JMS connection has
> failed: java.io.EOFException
> at
> org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1448)
> at
> org.apache.activemq.ActiveMQConnection.doStop(ActiveMQConnection.java:580)
> at
> org.apache.activemq.ActiveMQConnection.stop(ActiveMQConnection.java:569)
> at
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer$3.run(UimaDefaultMessageListenerContainer.java:1134)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:403)
> at
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
> at
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
> at java.lang.Thread.run(Thread.java:780)
>
> 12:21:48.687 - 23:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
> to Broker: \
> tcp://localhost:61616 - Ready to Process Again
> 12:21:48.709 - 36:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
> to Broker: \
> tcp://localhost:61616 - Ready to Process Again
> 12:21:48.814 - 27:
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue(1059):
> INFO: Test Aggregate
> TAE-JmsInputChannel.createListenerOnTempQueue()-starting new Listener
> 12:21:48.866 - 27:
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue:
> INFO: Service:Test Aggregate TAE Unable to refresh temp destination -
> retrying in 5 seconds until success \
> ...
> 12:21:53.869 - 27:
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue:
> INFO: Service:Test Aggregate TAE succesfully refreshed temp
> destination:temp-queue://ID:bluejws65-33772-14\
> 78017289483-1:3:1 - FreeCas Queue:true
> 12:21:53.869 - 27:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
> to Broker: \
> tcp://localhost:61616 - Ready to Process Again
>
>
>
>
> When running the scenario can you please check if UIMA_HOME points to the
> new code (the snapshot). Perhaps it points to an older version of UIMA-AS.
> Not sure if this is the case but worth checking.
>
> Jerry
>
> On Tue, Nov 1, 2016 at 9:17 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> I tried the snapshot that you gave me, and I understand perfectly the
>> explanation of recovery queue that you explain, but to me not work,
>> and y get the same behavior. With the last snapshop, the uima-as log
>> never show recovery of new queue after restart broker (restart =
>> execute /bin/startBroker.sh Starting in foreground, then pressing
>> CTRL+C, then execute startBroker.sh again), only shows:
>>
>> 08:58:13.825 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint:
>> ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
>> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 08:58:13.825 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.handleTempQueueFailure:
>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
>> 08:58:13.825 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.handleTempQueueFailure:
>> WARNING: Jms Listener Failed. Endpoint:
>> temp-queue://ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
>> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 08:58:13.825 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying ...
>> 08:58:13.839 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
>> 08:58:13.844 - 46:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 08:58:13.844 - 46:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying ...
>> 08:58:13.876 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
>> Established Connection to Broker: tcp://localhost:61616
>> 08:58:16.719 - 46:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
>> Established Connection to Broker: tcp://localhost:61616
>> 08:59:16.887 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.IllegalStateException: The Consumer is closed
>> 08:59:16.887 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying ...
>> 08:59:16.900 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
>> Established Connection to Broker: tcp://localhost:61616
>> 08:59:16.917 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
>> 08:59:16.917 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.IllegalStateException: The Consumer is closed
>> 08:59:16.917 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying ...
>> 08:59:16.948 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
>> Established Connection to Broker: tcp://localhost:61616
>> 09:01:17.93 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.IllegalStateException: The Consumer is closed
>> 09:01:17.93 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying ...
>> 09:01:17.109 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
>> Established Connection to Broker: tcp://localhost:61616
>>
>>
>>
>> And in the console output of service:
>> Could not refresh JMS Connection for destination
>> 'queue://XClusterAnalyzerAggregate' - silently retrying in 5 ms.
>> Cause: Could not connect to broker URL: tcp://localhost:61616. Reason:
>> java.net.ConnectException: Connection refused
>>
>> 2016-10-27 14:29 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Nelson, try this fix:
>> > https://repository.apache.org/content/repositories/
>> snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-
>> as-2.9.0-20161027.181519-7.tar.gz
>> >
>> > Keep in mind that if you bounce a broker while a CAS is being processed
>> in
>> > a service, the client will not be able to get the input CAS back nor
>> will
>> > it get child CASes. Both the input and child CAS will be thrown away by
>> the
>> > service due to undeliverable destination (old temp queue). The code
>> > will
>> > actually stop the Cas Multiplier from producing child CASes if it
>> > detects
>> > an undeliverable destination. You need a process timeout on the client
>> > side to deal with that. When a broker is bounced, both the client and a
>> > service will detect a new broker instance, kill its old temp queue
>> > listeners and create new ones so that subsequent processing can take
>> place.
>> >
>> > Jerry
>> >
>> > On Thu, Oct 27, 2016 at 1:19 PM, nelson rivera
>> > <nelsonrivera12@gmail.com
>> >
>> > wrote:
>> >
>> >> Hi, I would like know if you have worked in the solution of this
>> >> theme, related with recovered of queues in service cas multiplier,
>> >> after restart uima-as broker, specifically temp queue. If you have a
>> >> snapshot with this. Thanks
>> >>
>> >> 2016-09-28 10:34 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > Nelson, yes I think there is a problem in the UIMA-AS. First, there
>> >> > is
>> >> > evidence that a service
>> >> > is trying to use a stale temp queue:
>> >> >
>> >> > tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
>> >> > Cannot use a Temporary destination from another Connection.
>> >> >
>> >> >
>> >> > Also, I think there is another problem related to recovery of temp
>> >> > queues
>> >> > in the service.
>> >> > The service cas multiplier uses multiple queues:
>> >> > 1) input queue - receives GetMeta and Process requests from a client
>> >> > 2) temp queue - receives FreeCas requests from a client. This is
>> >> > only
>> >> > created when a service
>> >> > is a Cas Multiplier. This is used to free CASes in a cas
>> >> > multiplier.
>> >> >
>> >> > The input queue listener recovers the connection since it received
>> >> > an
>> >> input
>> >> > CAS. I think the
>> >> > temp queue is not recovered and this leads to repeated retries.
>> >> >
>> >> > I will work on fixing this today and let you know when this is
>> >> > ready.
>> I
>> >> > will also create a new
>> >> > snapshot for you to try.
>> >> >
>> >> > Thanks for finding these problems.
>> >> >
>> >> > - jerry
>> >> >
>> >> >
>> >> > On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <
>> >> nelsonrivera12@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> ok, only show you the result of last test.
>> >> >> Now in my test with uima-as-2.9.0, after complete restart of the
>> >> >> Broker UIMA-AS log show
>> >> >>
>> >> >> 09:51:01.731 - 19:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.
>> >> >> handleListenerSetupFailure:
>> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
>> Recovered
>> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process
>> >> >> Again
>> >> >> 09:51:01.740 - 46:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.
>> >> >> handleListenerSetupFailure:
>> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
>> Recovered
>> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process
>> >> >> Again
>> >> >> 09:51:01.747 - 15:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.
>> >> >> handleListenerSetupFailure:
>> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
>> Recovered
>> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process
>> >> >> Again
>> >> >> 09:51:01.773 - 14:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> >> ainer.onException:
>> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> >> 09:51:01.773 - 14:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> >> ainer.onException:
>> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> >> 09:51:01.774 - 14:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> >> ainer.onException:
>> >> >> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >> java.io.EOFException
>> >> >> 09:51:01.774 - 14:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.
>> >> >> handleListenerSetupFailure:
>> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable
>> To
>> >> >> Connect To Broker: tcp://localhost:61616 Retrying Until Successful
>> ...
>> >> >> 09:51:01.777 - 19:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> >> ainer.onException:
>> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> >> 09:51:01.777 - 19:
>> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> >> ainer.onException:
>> >> >> WARNING: Jms Listener Failed. Endpoint:
>> >> >> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
>> >> >> tcp://localhost:61616 Reason:
>> >> >> javax.jms.InvalidDestinationException:
>> >> >> Cannot use a Temporary destination from another Connection.
>> >> >>
>> >> >> this lines are continue to repeat in the log indefinitely. And
>> >> >> after
>> >> >> this have the same problem when send input Cas at the service
>> >> >>
>> >> >> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >> > Nelson, I will investigate your use case and let you know.
>> >> >> >
>> >> >> > -jerry
>> >> >> >
>> >> >> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera
>> >> >> > <nelsonrivera12@gmail.com
>> >> >> >
>> >> >> > wrote:
>> >> >> >
>> >> >> >> Thank you for describe how UIMA-AS works, it is very helpful.
>> >> >> >> But
>> >> >> >> my
>> >> >> >> scenario is not restart the broker while the uima-as service
>> >> >> >> works
>> >> >> >> on
>> >> >> >> input CAS
>> >> >> >>
>> >> >> >> my scenario is
>> >> >> >> 1- Start Broker and UIMA-AS aggregate service
>> >> >> >>
>> >> >> >> 2- Complete Restart of the Broker (without restart the
>> >> >> >> service).
>> >> >> >> UIMA-AS log show the lines that previously i said you:
>> >> >> >> "Jms Listener Failed." ....
>> >> >> >> "The Session is closed." ...
>> >> >> >> "The Consumer is closed." ...
>> >> >> >>
>> >> >> >> I do not see in my uima-as log the lines that you say in your
>> >> >> >> test
>> >> >> >> after broker is bounced:
>> >> >> >> Listeners reconnect:
>> >> >> >> INFO | Successfully refreshed JMS Connection
>> >> >> >> INFO | Successfully refreshed JMS Connection
>> >> >> >> In my case (uima-as 2.8.1) seems to be a problem with
>> >> >> >> reestablishing
>> >> >> >> the connection to the broker after restart.
>> >> >> >>
>> >> >> >> 4- Then in this point, after the broker was restarted
>> >> >> >> completely,
>> >> >> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for
>> >> >> >> this
>> >> >> >> petition UIMA-AS client receive the parent CAS, but not child
>> >> >> >> CAS.
>> >> >> >>
>> >> >> >> In this moment i will trys with uima-as-2.9.0 snapshot
>> >> >> >>
>> >> >> >>
>> >> >> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >> >> > Nelson, in my scenario I do have a Cas Multiplier which
>> >> >> >> > produces
>> >> >> >> > a
>> >> >> >> > child
>> >> >> >> > CAS for each input CAS. For the purpose of this
>> >> >> >> > test its not important how many internal delegates there are
>> >> >> >> > in
>> a
>> >> >> >> pipeline.
>> >> >> >> > The service is configured as a Cas Multiplier
>> >> >> >> > and it does send child CASes when a broker is running.
>> >> >> >> >
>> >> >> >> > The problem is that once the input CAS is received, its reply
>> >> >> >> > queue
>> >> >> >> > is
>> >> >> >> > bound to a specific temp reply queue. When a broker is
>> >> >> >> > bounced, that queue is not valid anymore and cant be used to
>> >> >> >> > return
>> >> >> >> > child
>> >> >> >> > CASes nor the input CAS. To guard against such
>> >> >> >> > cases you need to use timeouts on the client side.
>> >> >> >> >
>> >> >> >> > Like I said in my previous email I dont get how an input CAS
>> >> >> >> > can
>> >> >> >> > be
>> >> >> >> > returned back to the client after a broker bounce. This should
>> >> >> >> > not happen since its temp queue does not exist in a new
>> >> >> >> > broker.
>> >> >> >> >
>> >> >> >> > I am still looking at this scenario as I don't like what I see
>> in
>> >> >> >> > the
>> >> >> >> > service log. I think there might be a bug in the UIMA-AS but I
>> am
>> >> >> >> > still investigating. Bug or not, what I described above is how
>> >> >> >> > UIMA-AS
>> >> >> >> > works. You should not expect any replies from a service
>> >> >> >> > working on a CAS that was sent through a broker that is
>> >> >> >> > bounced
>> >> >> >> > before
>> >> >> >> > a
>> >> >> >> > reply.
>> >> >> >> >
>> >> >> >> > -jerry
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
>> >> >> >> > <nelsonrivera12@gmail.com
>> >> >> >> >
>> >> >> >> > wrote:
>> >> >> >> >
>> >> >> >> >> My service is a aggregate analysis engine that receives
>> >> >> >> >> input
>> >> >> >> >> CAS
>> >> >> >> >> with the direcction of a directory of documents, the first
>> >> >> >> >> component
>> >> >> >> >> is a MultiplerCas that for each document generate a child
>> >> >> >> >> CAS,
>> >> each
>> >> >> >> >> child CAS continues the flow that are other primitive
>> annotators
>> >> >> >> >> (primitive annotators in c++ that can throw exceptions), then
>> >> >> >> >> exists
>> >> >> >> >> other MultiplerCas that acts as Merger, waiting the complete
>> >> >> >> >> processing of each child CAS and finally generate a child CAS
>> >> where
>> >> >> >> >> each child CAS previously processed is a view of CAS, this a
>> >> >> >> >> child
>> >> >> CAS
>> >> >> >> >> that is composed of view continúa el flujo a otro AE, que
>> espera
>> >> un
>> >> >> >> >> CAS de esta manera.
>> >> >> >> >> I will send you my uima-as descriptor, to understand better
>> >> >> >> >> my
>> >> >> >> >> service. All delegates are co-located.
>> >> >> >> >>
>> >> >> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see
>> >> >> >> >> how
>> >> >> >> >> it
>> >> >> >> >> behaves
>> >> >> >> >>
>> >> >> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik
>> >> >> >> >> <uimaee@gmail.com
>> >:
>> >> >> >> >> > I was able to stage this scenario in which an aggregate
>> >> >> >> >> > service
>> >> >> >> >> > cas
>> >> >> >> >> > multiplier receives a single input CAS and
>> >> >> >> >> > produces a child CAS.
>> >> >> >> >> >
>> >> >> >> >> > Scenario:
>> >> >> >> >> > 1) Service receives input CAS
>> >> >> >> >> > 2) CasMultiplier generates a child CAS
>> >> >> >> >> > 3) AE.process() is called (child CAS) and it sleeps for
>> awhile
>> >> >> >> >> > 4) Broker is shutdown
>> >> >> >> >> > 5) AE.process() awakes from sleep
>> >> >> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
>> >> >> >> >> > 7) Connection is invalid so the child CAS is dropped
>> >> >> >> >> > 8) Broker is restarted
>> >> >> >> >> > 9) Listeners reconnect:
>> >> >> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> >> >> > 10) Broker log shows:
>> >> >> >> >> > WARN Service - Async error
>> >> >> >> >> > occurred:
>> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> >> >> > destination
>> >> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does
>> >> >> >> >> > not
>> >> >> exist.
>> >> >> >> >> > WARN Service - Async error
>> >> >> >> >> > occurred:
>> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> >> >> > destination
>> >> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does
>> >> >> >> >> > not
>> >> >> exist.
>> >> >> >> >> >
>> >> >> >> >> > The service failed to send a reply.Nothing is delivered to
>> >> >> >> >> > a
>> >> >> >> >> > client
>> >> >> >> >> > as
>> >> >> >> >> > expected. This is the confusing part since you see a parent
>> >> >> >> >> > CAS
>> >> >> >> >> > being
>> >> >> >> >> sent
>> >> >> >> >> > to a client. I just cant imagine how this is possible.
>> >> >> >> >> >
>> >> >> >> >> > When a service receives an input CAS in a message, it
>> extracts
>> >> >> >> >> > a
>> >> >> JMS
>> >> >> >> >> > replyTo object which is a JMS temp queue and stores in an
>> >> >> >> >> > internal
>> >> >> >> >> > cache
>> >> >> >> >> > until it is time to send a reply. Temp queues are transient
>> >> >> >> >> > and
>> >> >> >> >> > unique
>> >> >> >> >> each
>> >> >> >> >> > time one is created.
>> >> >> >> >> >
>> >> >> >> >> > If the broker dies and subsequently restarts, the cached
>> >> >> >> >> > replyTo
>> >> >> >> >> > queue
>> >> >> >> >> > object is invalid (temp queue no longer exist) which leads
>> >> >> >> >> > to
>> >> >> >> >> > "Async error occurred:
>> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException" as
>> shows
>> >> >> >> >> > above.
>> >> >> >> >> >
>> >> >> >> >> > In the service log I see a bunch of these exceptions:
>> >> >> >> >> >
>> >> >> >> >> > 02:05:46.574 - 56:
>> >> >> >> >> > org.apache.uima.adapter.jms.activemq.
>> >> JmsEndpointConnection_impl.
>> >> >> >> >> openChannel:
>> >> >> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open
>> Connection
>> >> To
>> >> >> >> >> > Broker:
>> >> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
>> >> >> Silently
>> >> >> >> >> > Retrying ...
>> >> >> >> >> > 02:05:46.574 - 56:
>> >> >> >> >> > org.apache.uima.adapter.jms.activemq.
>> >> JmsEndpointConnection_impl.
>> >> >> >> >> openChannel:
>> >> >> >> >> > WARNING:
>> >> >> >> >> > javax.jms.JMSException: Could not connect to broker URL:
>> >> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0.
>> >> Reason:
>> >> >> >> >> > java.net.ConnectException: Connection refused
>> >> >> >> >> >
>> >> >> >> >> > This is *after* the service is restarted so there seems to
>> >> >> >> >> > be
>> >> >> >> >> > a
>> >> >> >> problem
>> >> >> >> >> > with reestablishing the connection to a new broker.
>> >> >> >> >> > I am still investigating that part. I just am confused how
>> you
>> >> >> >> >> > are
>> >> >> >> able
>> >> >> >> >> to
>> >> >> >> >> > see a reply in the client.
>> >> >> >> >> >
>> >> >> >> >> > -Jerry
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
>> >> >> uimaee@gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> >
>> >> >> >> >> >> From what I understand there are two issues:
>> >> >> >> >> >>
>> >> >> >> >> >> 1) Why service JMS listeners are not reconnecting when a
>> >> >> >> >> >> broker
>> >> >> >> >> >> is
>> >> >> >> >> >> bounced. This
>> >> >> >> >> >> should be done by default. The 2.9.0 snapshot may
>> address
>> >> >> >> >> >> this
>> >> >> >> >> >> since
>> >> >> >> >> >> a
>> >> >> >> >> >> listener
>> >> >> >> >> >> uses newer AMQ jars (5.14.0)
>> >> >> >> >> >>
>> >> >> >> >> >> 2) What happens to a child CAS when a Broker disappears
>> >> >> >> >> >> and
>> >> >> >> >> >> the
>> >> >> >> >> >> service
>> >> >> >> >> >> wants
>> >> >> >> >> >> to deliver it to a client that is not reachable. I
>> >> >> >> >> >> need
>> >> >> >> >> >> to
>> >> >> >> >> >> investigate
>> >> >> >> >> >> this part. I think
>> >> >> >> >> >> such CAS is dropped but that should be reflected in a
>> log
>> >> >> >> >> >> and
>> >> >> >> also
>> >> >> >> >> >> the
>> >> >> >> >> >> input CAS
>> >> >> >> >> >> should be marked as failed. Again something for me to
>> >> >> >> investigate.
>> >> >> >> >> >>
>> >> >> >> >> >> I will try to set up a test to see if I can create the
>> >> >> >> >> >> scenario.It
>> >> >> >> >> >> will
>> >> >> >> >> >> take me a bit of time
>> >> >> >> >> >> to set this up but I will let you know what happens.
>> >> >> >> >> >>
>> >> >> >> >> >> -jerry
>> >> >> >> >> >>
>> >> >> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
>> >> >> >> >> >> <uimaee@gmail.com
>> >> >> >> >
>> >> >> >> >> >> wrote:
>> >> >> >> >> >>
>> >> >> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> >> >> >> >> >>> https://repository.apache.org/
>> content/repositories/snapshots
>> >> >> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-
>> 201609
>> >> >> >> >> >>> 23.195233-6.tar.gz
>> >> >> >> >> >>>
>> >> >> >> >> >>> Can you briefly describe your service? Does your service
>> >> >> >> >> >>> have
>> >> >> >> >> >>> any
>> >> >> >> >> remote
>> >> >> >> >> >>> delegates or all parts of the pipeline are in the same
>> >> process?
>> >> >> >> >> >>> Do you see and exceptions in the service log?
>> >> >> >> >> >>>
>> >> >> >> >> >>> -jerry
>> >> >> >> >> >>>
>> >> >> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>> >> >> >> >> >>> <ne...@gmail.com>
>> >> >> >> >> >>> wrote:
>> >> >> >> >> >>>
>> >> >> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same
>> >> error.
>> >> >> >> >> >>>> not
>> >> >> >> >> >>>> appears to be broker
>> >> >> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i
>> >> >> >> >> >>>> am
>> >> >> >> >> >>>> stagnant
>> >> >> >> >> >>>> at
>> >> >> >> >> >>>> work
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <
>> >> uimaee@gmail.com
>> >> >> >:
>> >> >> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is
>> >> >> >> >> >>>> > related
>> >> >> >> >> >>>> > to
>> >> >> >> >> broker
>> >> >> >> >> >>>> > issues.
>> >> >> >> >> >>>> >
>> >> >> >> >> >>>> > -jerry
>> >> >> >> >> >>>> >
>> >> >> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> >> >> >> >> >>>> nelsonrivera12@gmail.com>
>> >> >> >> >> >>>> > wrote:
>> >> >> >> >> >>>> >
>> >> >> >> >> >>>> >> just like that, the parent CAS comes back but the
>> >> >> >> >> >>>> >> child
>> >> CAS
>> >> >> >> >> >>>> >> doesnt.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> And the child CAS if processed, because the uima-as
>> >> >> >> >> >>>> >> log
>> >> >> >> >> >>>> >> shows
>> >> >> >> >> >>>> >> the
>> >> >> >> >> >>>> >> lines of the other annotators primitive of the flow
>> >> >> processing
>> >> >> >> >> >>>> >> the
>> >> >> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> uima-as log only shows
>> >> >> >> >> >>>> >> "Jms Listener Failed. Endpoint:
>> >> >> >> >> >>>> >> temp-queue://ID:localhsot-H81-
>> >> M1-33610-1474481197190-1:1:1
>> >> >> >> >> >>>> >> Managed
>> >> >> >> >> >>>> By:
>> >> >> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >> >> >> >>>> >> java.io.EOFException" ........
>> >> >> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >> >> >>>> >> XClusterAnalyzerAggregate
>> >> >> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
>> >> >> >> >> >>>> >> closed..............
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> The CAS is send to service after restart the Broker,
>> and
>> >> >> >> >> >>>> >> how
>> >> >> >> >> >>>> >> say
>> >> >> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And
>> >> >> >> >> >>>> >> if
>> i
>> >> >> >> >> >>>> >> try
>> >> >> >> >> quiesce
>> >> >> >> >> >>>> >> the Aggregate Service console shows : "Quiescing
>> UIMA-AS
>> >> >> >> Service.
>> >> >> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose
>> that
>> >> is
>> >> >> >> >> >>>> >> the
>> >> >> >> >> >>>> >> child
>> >> >> >> >> >>>> >> CAS.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> the client if can send more CASes to the service, but
>> >> >> >> >> >>>> >> not
>> >> >> >> receive
>> >> >> >> >> >>>> >> response, for each CAS send after the one CAS that
>> >> >> >> >> >>>> >> was
>> >> send
>> >> >> >> after
>> >> >> >> >> >>>> >> restart broker (for which parent CAS comes back but
>> the
>> >> >> child
>> >> >> >> >> >>>> >> CAS
>> >> >> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> More specific:
>> >> >> >> >> >>>> >> scenario 1:
>> >> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
>> >> >> >> >> >>>> >> Everything
>> >> >> >> >> >>>> >> is
>> >> >> >> >> >>>> >> fine.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
>> >> >> >> >> >>>> >> UIMA-AS
>> >> >> >> >> >>>> >> log
>> >> >> >> >> >>>> >> show
>> >> >> >> >> >>>> >> :
>> >> >> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate
>> >> >> >> >> >>>> >> service.(The
>> >> >> parent
>> >> >> >> >> >>>> >> CAS
>> >> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
>> >> >> >> >> >>>> >> client.)
>> >> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows
>> >> >> >> >> >>>> >> :
>> >> >> >> "Quiescing
>> >> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to
>> Process:1"
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> scenario 2:
>> >> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
>> >> >> >> >> >>>> >> Everything
>> >> >> >> >> >>>> >> is
>> >> >> >> >> >>>> >> fine.
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
>> >> >> >> >> >>>> >> UIMA-AS
>> >> >> >> >> >>>> >> log
>> >> >> >> >> >>>> >> show
>> >> >> >> >> >>>> >> :
>> >> >> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate
>> >> >> >> >> >>>> >> service.(The
>> >> >> parent
>> >> >> >> >> >>>> >> CAS
>> >> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
>> >> client)
>> >> >> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
>> >> >> service.(not
>> >> >> >> >> comes
>> >> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
>> >> >> service.(not
>> >> >> >> >> comes
>> >> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >> >> >>>> >> If i try quiesce the Aggregate Service console
>> >> >> >> >> >>>> >> shows:
>> >> >> >> >> >>>> >> "Quiescing
>> >> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to
>> Process:5"
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
>> >> >> uimaee@gmail.com
>> >> >> >> >:
>> >> >> >> >> >>>> >> > When a parent CAS comes back (after broker is
>> >> >> >> >> >>>> >> > restored),
>> >> >> can
>> >> >> >> >> >>>> >> > the
>> >> >> >> >> >>>> client
>> >> >> >> >> >>>> >> > send more CASes to the service and receive
>> >> >> >> >> >>>> >> > responses?
>> >> >> >> >> >>>> >> > I
>> >> >> >> >> >>>> >> > am
>> >> >> >> >> >>>> >> > trying
>> >> >> >> >> >>>> >> > to
>> >> >> >> >> >>>> >> > determine what the real problem is here.
>> >> >> >> >> >>>> >> >
>> >> >> >> >> >>>> >> > 1) JMS Listener not recovering connection
>> >> >> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when
>> >> broker
>> >> >> >> >> >>>> connection is
>> >> >> >> >> >>>> >> > stale
>> >> >> >> >> >>>> >> >
>> >> >> >> >> >>>> >> > -jerry
>> >> >> >> >> >>>> >> >
>> >> >> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik
>> >> >> >> >> >>>> >> > <
>> >> >> >> >> >>>> uimaee@gmail.com>
>> >> >> >> >> >>>> >> > wrote:
>> >> >> >> >> >>>> >> >
>> >> >> >> >> >>>> >> >> Strange. You say that the parent CAS comes back
>> >> >> >> >> >>>> >> >> but
>> >> >> >> >> >>>> >> >> the
>> >> >> >> >> >>>> >> >> child
>> >> >> >> >> CAS
>> >> >> >> >> >>>> >> doesnt.
>> >> >> >> >> >>>> >> >> Are there any exceptions in the service log?
>> >> >> >> >> >>>> >> >> Perhaps
>> >> the
>> >> >> >> >> >>>> >> >> service
>> >> >> >> >> >>>> >> >> is
>> >> >> >> >> >>>> >> >> unable
>> >> >> >> >> >>>> >> >> to send the child CAS due to missing broker. There
>> >> >> >> >> >>>> >> >> should
>> >> >> >> >> >>>> >> >> be
>> >> >> >> >> >>>> something
>> >> >> >> >> >>>> >> in
>> >> >> >> >> >>>> >> >> the uima-as log about this.
>> >> >> >> >> >>>> >> >>
>> >> >> >> >> >>>> >> >> -jerry
>> >> >> >> >> >>>> >> >>
>> >> >> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> >> >> >> >>>> >> >> <ne...@gmail.com>
>> >> >> >> >> >>>> >> >> wrote:
>> >> >> >> >> >>>> >> >>
>> >> >> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
>> >> >> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS
>> >> >> >> >> >>>> >> >>> continue
>> >> >> >> >> >>>> >> >>> in
>> >> >> >> >> >>>> >> >>> the
>> >> >> >> >> >>>> >> >>> flow,
>> >> >> >> >> >>>> >> >>> that are other primitive annotators. And I
>> >> >> >> >> >>>> >> >>> expected
>> >> >> >> >> >>>> >> >>> that
>> >> >> >> >> >>>> >> >>> the
>> >> >> >> >> >>>> >> >>> child
>> >> >> >> >> >>>> >> >>> CAS
>> >> >> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>> the scenario:
>> >> >> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >> >> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate
>> >> >> >> >> >>>> >> >>> service
>> >> >> >> >> >>>> >> >>> to
>> >> >> >> >> >>>> >> >>> verify
>> >> >> >> >> >>>> >> >>> everything is fine. The process is correct.
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>> 3- Restart Broker (without restart the service).
>> >> >> >> >> >>>> >> >>> UIMA-AS
>> >> >> >> log
>> >> >> >> >> >>>> show :
>> >> >> >> >> >>>> >> >>> "Jms Listener Failed." ....
>> >> >> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate
>> >> >> >> >> >>>> >> >>> service
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not
>> child
>> >> >> >> >> >>>> >> >>> CAS.
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>> Thank for you quick reply
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
>> >> >> >> uimaee@gmail.com
>> >> >> >> >> >:
>> >> >> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
>> >> >> >> Multiplier.
>> >> >> >> >> >>>> Meaning
>> >> >> >> >> >>>> >> >>> given a
>> >> >> >> >> >>>> >> >>> > CAS it produces child CASes
>> >> >> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this
>> >> >> >> >> >>>> >> >>> > what
>> >> >> this
>> >> >> >> >> >>>> service is
>> >> >> >> >> >>>> >> >>> > supposed to do if the broker is running?
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> > Can you confirm the scenario:
>> >> >> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate
>> service
>> >> >> >> >> >>>> >> >>> > 2) Service receives the CAS
>> >> >> >> >> >>>> >> >>> > 3) Broker dies
>> >> >> >> >> >>>> >> >>> > 4) Broker is restarted
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in
>> >> >> >> >> >>>> >> >>> > a
>> >> >> >> >> >>>> >> >>> > reply
>> >> >> >> >> >>>> >> >>> > when
>> >> >> >> >> a
>> >> >> >> >> >>>> >> >>> > broker
>> >> >> >> >> >>>> >> >>> > is
>> >> >> >> >> >>>> >> >>> > restarted?
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> > -Jerry
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera
>> >> >> >> >> >>>> >> >>> > <
>> >> >> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >> >> >>>> >> >>> > wrote:
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>> >> I will try the variants that you say. I was
>> >> >> >> >> >>>> >> >>> >> deepening
>> >> >> >> >> >>>> >> >>> >> the
>> >> >> >> >> >>>> problem:
>> >> >> >> >> >>>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> I have a Aggregate AE that contains
>> >> >> >> >> >>>> >> >>> >> CasMultiplier,
>> >> >> >> >> >>>> >> >>> >> the
>> >> >> >> >> >>>> >> >>> >> original
>> >> >> >> >> >>>> >> >>> >> CAS
>> >> >> >> >> >>>> >> >>> >> is finish when it arrives to this
>> >> >> >> >> >>>> >> >>> >> CasMultiplier,
>> >> >> >> >> >>>> >> >>> >> this
>> >> >> >> >> >>>> generates
>> >> >> >> >> >>>> >> >>> >> new
>> >> >> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow
>> >> >> >> >> >>>> >> >>> >> of
>> >> >> >> >> >>>> >> >>> >> Aggregate.
>> >> >> >> >> >>>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> After restart broker (without restart the
>> service
>> >> of
>> >> >> >> >> >>>> >> >>> >> annotator)
>> >> >> >> >> >>>> >> >>> >> and
>> >> >> >> >> >>>> >> >>> >> send a CAS to process, the listener
>> >> >> >> >> >>>> >> >>> >> associated
>> >> >> >> >> >>>> >> >>> >> to
>> >> >> >> >> >>>> >> >>> >> the
>> >> >> >> >> client
>> >> >> >> >> >>>> api
>> >> >> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is
>> >> >> >> >> >>>> >> >>> >> notified
>> >> >> >> >> >>>> >> >>> >> when
>> >> >> >> >> >>>> the
>> >> >> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas
>> generated
>> >> in
>> >> >> >> >> >>>> >> >>> >> the
>> >> >> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when
>> finalizes.
>> >> >> >> >> >>>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same,
>> but
>> >> >> >> >> >>>> >> >>> >> how
>> >> >> >> >> >>>> >> >>> >> the
>> >> >> >> >> >>>> original
>> >> >> >> >> >>>> >> >>> >> CAS
>> >> >> >> >> >>>> >> >>> >> is notifies into the listener when finalizes,
>> >> >> >> >> >>>> >> >>> >> not
>> >> >> have
>> >> >> >> >> >>>> >> >>> >> problem
>> >> >> >> >> >>>> >> >>> >> because is not expected for another new CAS.
>> >> >> >> >> >>>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>> >> >> >> >> >>>> >> >>> >> <uimaee@gmail.com
>> >> >> >> >> >>>> >:
>> >> >> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0
>> >> >> >> >> >>>> >> >>> >> > (latest)
>> >> >> >> >> >>>> >> >>> >> > and
>> >> >> >> use
>> >> >> >> >> >>>> >> >>> >> > that
>> >> >> >> >> >>>> >> >>> >> > broker
>> >> >> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships
>> >> >> >> >> >>>> >> >>> >> > with?
>> >> >> >> >> >>>> >> >>> >> > There
>> >> >> >> >> >>>> >> >>> >> > was
>> >> >> >> >> >>>> >> >>> >> > a
>> >> >> >> >> >>>> >> >>> >> > nasty
>> >> >> >> >> >>>> >> >>> bug
>> >> >> >> >> >>>> >> >>> >> > in
>> >> >> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not
>> working.
>> >> >> >> >> >>>> >> >>> >> > I
>> >> >> >> >> >>>> >> >>> >> > am
>> >> >> >> not
>> >> >> >> >> >>>> sure if
>> >> >> >> >> >>>> >> >>> >> > this
>> >> >> >> >> >>>> >> >>> >> > is
>> >> >> >> >> >>>> >> >>> >> > the cause but worth trying.
>> >> >> >> >> >>>> >> >>> >> >
>> >> >> >> >> >>>> >> >>> >> > You can download AMQ from here:
>> >> >> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
>> >> >> >> >> >>>> >> >>> >> >
>> >> >> >> >> >>>> >> >>> >> > -Jerry
>> >> >> >> >> >>>> >> >>> >> >
>> >> >> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw
>> >> >> >> >> >>>> >> >>> >> > Cwiklik
>> >> >> >> >> >>>> >> >>> >> > <ui...@gmail.com>
>> >> >> >> >> >>>> >> >>> >> wrote:
>> >> >> >> >> >>>> >> >>> >> >
>> >> >> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to
>> stage
>> >> >> >> >> >>>> >> >>> >> >> this
>> >> >> >> >> >>>> >> >>> >> >> with
>> >> >> >> >> >>>> >> >>> >> >> UIMA-AS
>> >> >> >> >> >>>> >> >>> >> >> 2.8.1.
>> >> >> >> >> >>>> >> >>> >> I
>> >> >> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest)
>> code.
>> >> >> >> >> >>>> >> >>> >> >> The
>> >> >> >> most
>> >> >> >> >> >>>> current
>> >> >> >> >> >>>> >> >>> >> >> code
>> >> >> >> >> >>>> >> >>> >> does
>> >> >> >> >> >>>> >> >>> >> >> have newer broker.
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >> >> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> After the service started, I stopped the
>> >> >> >> >> >>>> >> >>> >> >> broker,
>> >> >> >> waited
>> >> >> >> >> >>>> >> >>> >> >> a
>> >> >> >> >> >>>> bit,
>> >> >> >> >> >>>> >> and
>> >> >> >> >> >>>> >> >>> >> >> restarted the broker.
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following
>> in
>> >> >> >> >> >>>> >> >>> >> >> the
>> >> >> >> log:
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1:
>> org.apache.uima.adapter.jms.
>> >> >> >> activemq.
>> >> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
>> >> >> >> >> >>>> >> >>> >> >> INFO:
>> >> >> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002
>> Broker:
>> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1:
>> org.apache.uima.adapter.jms.
>> >> >> >> activemq.
>> >> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
>> >> >> >> >> >>>> >> >>> >> >> INFO:
>> >> >> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
>> >> >> tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> >> >> >> WARNING:
>> >> >> >> >> >>>> >> Service:
>> >> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> >> >> >> WARNING:
>> >> >> >> >> >>>> Jms
>> >> >> >> >> >>>> >> >>> Listener
>> >> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
>> >> Managed
>> >> >> >> >> >>>> >> >>> >> >> By:
>> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> >> >> java.io.EOFException
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE
>> >> >> >> >> >>>> >> >>> >> >> Listener
>> >> >> Unable
>> >> >> >> To
>> >> >> >> >> >>>> Connect
>> >> >> >> >> >>>> >> To
>> >> >> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
>> >> >> >> >> >>>> >> >>> >> >> Successful
>> >> >> >> >> >>>> >> >>> >> >> ...
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> >> >> >> WARNING:
>> >> >> >> >> >>>> >> Service:
>> >> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> >> >> >> WARNING:
>> >> >> >> >> >>>> Jms
>> >> >> >> >> >>>> >> >>> Listener
>> >> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
>> >> Managed
>> >> >> >> >> >>>> >> >>> >> >> By:
>> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> >> >> java.io.EOFException
>> >> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE
>> >> >> >> >> >>>> >> >>> >> >> Listener
>> >> >> Unable
>> >> >> >> To
>> >> >> >> >> >>>> Connect
>> >> >> >> >> >>>> >> To
>> >> >> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
>> >> >> >> >> >>>> >> >>> >> >> Successful
>> >> >> >> >> >>>> >> >>> >> >> ...
>> >> >> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE
>> >> >> >> >> >>>> >> >>> >> >> Listener
>> >> >> >> Recovered
>> >> >> >> >> >>>> >> Connection
>> >> >> >> >> >>>> >> >>> to
>> >> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
>> >> >> >> >> >>>> >> >>> >> >> Process
>> >> >> >> >> >>>> >> >>> >> >> Again
>> >> >> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
>> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE
>> >> >> >> >> >>>> >> >>> >> >> Listener
>> >> >> >> Recovered
>> >> >> >> >> >>>> >> Connection
>> >> >> >> >> >>>> >> >>> to
>> >> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
>> >> >> >> >> >>>> >> >>> >> >> Process
>> >> >> >> >> >>>> >> >>> >> >> Again
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> -Jerry
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson
>> rivera
>> >> >> >> >> >>>> >> >>> >> >> <
>> >> >> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
>> >> >> >> >> >>>> >> >>> >> >> wrote:
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that
>> >> >> >> >> >>>> >> >>> >> >>> what
>> >> >> happen?
>> >> >> >> >> >>>> >> >>> >> >>>
>> >> >> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw
>> >> >> >> >> >>>> >> >>> >> >>> Cwiklik
>> >> >> >> >> >>>> >> >>> >> >>> <
>> >> >> >> >> >>>> uimaee@gmail.com
>> >> >> >> >> >>>> >> >:
>> >> >> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >> >>>> >> >>> >> >>> > -Jerry
>> >> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson
>> >> rivera
>> >> >> >> >> >>>> >> >>> >> >>> > <
>> >> >> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >> >> >>>> >> >>> >> >>> > wrote:
>> >> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an
>> >> >> >> >> >>>> >> >>> >> >>> >> annotator
>> >> >> >> >> >>>> >> >>> >> >>> >> as
>> >> >> >> >> >>>> >> >>> >> >>> >> service,
>> >> >> >> >> >>>> >> >>> >> >>> >> processing
>> >> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the
>> >> >> >> >> >>>> >> >>> >> >>> >> broker
>> >> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >> stopped
>> >> >> >> >> >>>> >> >>> >> >>> >> and
>> >> >> >> >> >>>> >> >>> >> >>> >> subsequently
>> >> >> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines
>> >> >> >> >> >>>> >> >>> >> >>> >> below
>> >> >> and
>> >> >> >> >> after
>> >> >> >> >> >>>> this
>> >> >> >> >> >>>> >> >>> >> >>> >> the
>> >> >> >> >> >>>> >> >>> >> >>> >> api
>> >> >> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the
>> >> >> >> >> >>>> >> >>> >> >>> >> listener
>> >> >> >> when
>> >> >> >> >> >>>> >> >>> >> >>> >> a
>> >> >> >> >> >>>> cas
>> >> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >> >> >>>> >> >>> >> >>> >> processed, any more.
>> >> >> >> >> >>>> >> >>> >> >>> >> can help me?:
>> >> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> >> Analyzer
>> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >> >>>> >> >>> Exception
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >>> >> Reason:
>> >> >> >> >> >>>> >> >>> javax.jms.JMSException:
>> >> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
>> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >> >> >>>> Retrying
>> >> >> >> >> >>>> >> >>> >> >>> >> ...
>> >> >> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
>> >> >> >> >> >>>> >> >>> >> >>> >> Failed.
>> >> >> >> >> >>>> Endpoint:
>> >> >> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> >> >> >> >> >>>> M1-33610-1474481197190-1:1:1
>> >> >> >> >> >>>> >> >>> Managed
>> >> >> >> >> >>>> >> >>> >> >>> >> By:
>> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
>> >> >> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
>> >> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
>> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> >> Analyzer
>> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >> >>>> >> >>> Exception
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >>> >> Reason:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.
>> >> ConnectionClosedException:
>> >> >> >> The
>> >> >> >> >> >>>> >> connection
>> >> >> >> >> >>>> >> >>> is
>> >> >> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> >> Analyzer
>> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >> >>>> >> >>> Exception
>> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616
>> >> >> >> >> >>>> >> >>> >> >>> >> Reason:
>> >> >> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The
>> >> Consumer
>> >> >> is
>> >> >> >> >> >>>> closed.
>> >> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >> >>>> >> >>> >> >>>
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >> >>>> >> >>> >> >
>> >> >> >> >> >>>> >> >>> >>
>> >> >> >> >> >>>> >> >>> >
>> >> >> >> >> >>>> >> >>>
>> >> >> >> >> >>>> >> >>
>> >> >> >> >> >>>> >> >>
>> >> >> >> >> >>>> >> >
>> >> >> >> >> >>>> >>
>> >> >> >> >> >>>> >
>> >> >> >> >> >>>>
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >> >
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >>
>> >
>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Nelson, using the same snapshot I see a different behavior of UIMA-AS
service when a broker is bounced:
12:21:35.452 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
WARNING: Service: Test Aggregate TAE Runtime Exception
12:21:35.452 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
WARNING: Jms Listener Failed. Endpoint:
temp-queue://ID:bluejws65-33772-1478017289483-1:1\
:1 Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
12:21:35.453 - 36:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Test Aggregate TAE Runtime Exception
12:21:35.453 - 36:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: SecondLevelTaeQueue Managed By:
tcp://localhost:61616 Reason\
: javax.jms.JMSException: java.io.EOFException
12:21:35.453 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Test Aggregate TAE Listener Unable To Connect To
Broker: tcp\
://localhost:61616 Retrying Until Successful ...
12:21:35.453 - 36:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Test Aggregate TAE Listener Unable To Connect To
Broker: tcp\
://localhost:61616 Retrying Until Successful ...
12:21:35.454 - 46:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer$3.destroy:
WARNING:
org.apache.activemq.ConnectionFailedException: The JMS connection has
failed: java.io.EOFException
at
org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1448)
at
org.apache.activemq.ActiveMQConnection.doStop(ActiveMQConnection.java:580)
at
org.apache.activemq.ActiveMQConnection.stop(ActiveMQConnection.java:569)
at
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer$3.run(UimaDefaultMessageListenerContainer.java:1134)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:403)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
at java.lang.Thread.run(Thread.java:780)
12:21:48.687 - 23:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
to Broker: \
tcp://localhost:61616 - Ready to Process Again
12:21:48.709 - 36:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
to Broker: \
tcp://localhost:61616 - Ready to Process Again
12:21:48.814 - 27:
org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue(1059):
INFO: Test Aggregate
TAE-JmsInputChannel.createListenerOnTempQueue()-starting new Listener
12:21:48.866 - 27:
org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue:
INFO: Service:Test Aggregate TAE Unable to refresh temp destination -
retrying in 5 seconds until success \
...
12:21:53.869 - 27:
org.apache.uima.adapter.jms.activemq.JmsInputChannel.createListenerOnTempQueue:
INFO: Service:Test Aggregate TAE succesfully refreshed temp
destination:temp-queue://ID:bluejws65-33772-14\
78017289483-1:3:1 - FreeCas Queue:true
12:21:53.869 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Test Aggregate TAE Listener Recovered Connection
to Broker: \
tcp://localhost:61616 - Ready to Process Again
When running the scenario can you please check if UIMA_HOME points to the
new code (the snapshot). Perhaps it points to an older version of UIMA-AS.
Not sure if this is the case but worth checking.
Jerry
On Tue, Nov 1, 2016 at 9:17 AM, nelson rivera <ne...@gmail.com>
wrote:
> I tried the snapshot that you gave me, and I understand perfectly the
> explanation of recovery queue that you explain, but to me not work,
> and y get the same behavior. With the last snapshop, the uima-as log
> never show recovery of new queue after restart broker (restart =
> execute /bin/startBroker.sh Starting in foreground, then pressing
> CTRL+C, then execute startBroker.sh again), only shows:
>
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint:
> ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.handleTempQueueFailure:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.handleTempQueueFailure:
> WARNING: Jms Listener Failed. Endpoint:
> temp-queue://ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:58:13.839 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:58:13.844 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.844 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:58:13.876 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:58:16.719 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:59:16.887 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 08:59:16.887 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:59:16.900 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:59:16.948 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 09:01:17.93 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 09:01:17.93 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 09:01:17.109 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
>
>
>
> And in the console output of service:
> Could not refresh JMS Connection for destination
> 'queue://XClusterAnalyzerAggregate' - silently retrying in 5 ms.
> Cause: Could not connect to broker URL: tcp://localhost:61616. Reason:
> java.net.ConnectException: Connection refused
>
> 2016-10-27 14:29 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Nelson, try this fix:
> > https://repository.apache.org/content/repositories/
> snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-
> as-2.9.0-20161027.181519-7.tar.gz
> >
> > Keep in mind that if you bounce a broker while a CAS is being processed
> in
> > a service, the client will not be able to get the input CAS back nor
> will
> > it get child CASes. Both the input and child CAS will be thrown away by
> the
> > service due to undeliverable destination (old temp queue). The code will
> > actually stop the Cas Multiplier from producing child CASes if it detects
> > an undeliverable destination. You need a process timeout on the client
> > side to deal with that. When a broker is bounced, both the client and a
> > service will detect a new broker instance, kill its old temp queue
> > listeners and create new ones so that subsequent processing can take
> place.
> >
> > Jerry
> >
> > On Thu, Oct 27, 2016 at 1:19 PM, nelson rivera <nelsonrivera12@gmail.com
> >
> > wrote:
> >
> >> Hi, I would like know if you have worked in the solution of this
> >> theme, related with recovered of queues in service cas multiplier,
> >> after restart uima-as broker, specifically temp queue. If you have a
> >> snapshot with this. Thanks
> >>
> >> 2016-09-28 10:34 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > Nelson, yes I think there is a problem in the UIMA-AS. First, there is
> >> > evidence that a service
> >> > is trying to use a stale temp queue:
> >> >
> >> > tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> >> > Cannot use a Temporary destination from another Connection.
> >> >
> >> >
> >> > Also, I think there is another problem related to recovery of temp
> >> > queues
> >> > in the service.
> >> > The service cas multiplier uses multiple queues:
> >> > 1) input queue - receives GetMeta and Process requests from a client
> >> > 2) temp queue - receives FreeCas requests from a client. This is only
> >> > created when a service
> >> > is a Cas Multiplier. This is used to free CASes in a cas
> >> > multiplier.
> >> >
> >> > The input queue listener recovers the connection since it received an
> >> input
> >> > CAS. I think the
> >> > temp queue is not recovered and this leads to repeated retries.
> >> >
> >> > I will work on fixing this today and let you know when this is ready.
> I
> >> > will also create a new
> >> > snapshot for you to try.
> >> >
> >> > Thanks for finding these problems.
> >> >
> >> > - jerry
> >> >
> >> >
> >> > On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <
> >> nelsonrivera12@gmail.com>
> >> > wrote:
> >> >
> >> >> ok, only show you the result of last test.
> >> >> Now in my test with uima-as-2.9.0, after complete restart of the
> >> >> Broker UIMA-AS log show
> >> >>
> >> >> 09:51:01.731 - 19:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.
> >> >> handleListenerSetupFailure:
> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
> Recovered
> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> >> 09:51:01.740 - 46:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.
> >> >> handleListenerSetupFailure:
> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
> Recovered
> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> >> 09:51:01.747 - 15:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.
> >> >> handleListenerSetupFailure:
> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener
> Recovered
> >> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> >> 09:51:01.773 - 14:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> >> ainer.onException:
> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> >> 09:51:01.773 - 14:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> >> ainer.onException:
> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> >> 09:51:01.774 - 14:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> >> ainer.onException:
> >> >> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >> java.io.EOFException
> >> >> 09:51:01.774 - 14:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.
> >> >> handleListenerSetupFailure:
> >> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable
> To
> >> >> Connect To Broker: tcp://localhost:61616 Retrying Until Successful
> ...
> >> >> 09:51:01.777 - 19:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> >> ainer.onException:
> >> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> >> 09:51:01.777 - 19:
> >> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> >> ainer.onException:
> >> >> WARNING: Jms Listener Failed. Endpoint:
> >> >> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
> >> >> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> >> >> Cannot use a Temporary destination from another Connection.
> >> >>
> >> >> this lines are continue to repeat in the log indefinitely. And after
> >> >> this have the same problem when send input Cas at the service
> >> >>
> >> >> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >> > Nelson, I will investigate your use case and let you know.
> >> >> >
> >> >> > -jerry
> >> >> >
> >> >> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera
> >> >> > <nelsonrivera12@gmail.com
> >> >> >
> >> >> > wrote:
> >> >> >
> >> >> >> Thank you for describe how UIMA-AS works, it is very helpful. But
> >> >> >> my
> >> >> >> scenario is not restart the broker while the uima-as service works
> >> >> >> on
> >> >> >> input CAS
> >> >> >>
> >> >> >> my scenario is
> >> >> >> 1- Start Broker and UIMA-AS aggregate service
> >> >> >>
> >> >> >> 2- Complete Restart of the Broker (without restart the service).
> >> >> >> UIMA-AS log show the lines that previously i said you:
> >> >> >> "Jms Listener Failed." ....
> >> >> >> "The Session is closed." ...
> >> >> >> "The Consumer is closed." ...
> >> >> >>
> >> >> >> I do not see in my uima-as log the lines that you say in your test
> >> >> >> after broker is bounced:
> >> >> >> Listeners reconnect:
> >> >> >> INFO | Successfully refreshed JMS Connection
> >> >> >> INFO | Successfully refreshed JMS Connection
> >> >> >> In my case (uima-as 2.8.1) seems to be a problem with
> >> >> >> reestablishing
> >> >> >> the connection to the broker after restart.
> >> >> >>
> >> >> >> 4- Then in this point, after the broker was restarted completely,
> >> >> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for
> >> >> >> this
> >> >> >> petition UIMA-AS client receive the parent CAS, but not child CAS.
> >> >> >>
> >> >> >> In this moment i will trys with uima-as-2.9.0 snapshot
> >> >> >>
> >> >> >>
> >> >> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >> >> > Nelson, in my scenario I do have a Cas Multiplier which produces
> >> >> >> > a
> >> >> >> > child
> >> >> >> > CAS for each input CAS. For the purpose of this
> >> >> >> > test its not important how many internal delegates there are in
> a
> >> >> >> pipeline.
> >> >> >> > The service is configured as a Cas Multiplier
> >> >> >> > and it does send child CASes when a broker is running.
> >> >> >> >
> >> >> >> > The problem is that once the input CAS is received, its reply
> >> >> >> > queue
> >> >> >> > is
> >> >> >> > bound to a specific temp reply queue. When a broker is
> >> >> >> > bounced, that queue is not valid anymore and cant be used to
> >> >> >> > return
> >> >> >> > child
> >> >> >> > CASes nor the input CAS. To guard against such
> >> >> >> > cases you need to use timeouts on the client side.
> >> >> >> >
> >> >> >> > Like I said in my previous email I dont get how an input CAS can
> >> >> >> > be
> >> >> >> > returned back to the client after a broker bounce. This should
> >> >> >> > not happen since its temp queue does not exist in a new broker.
> >> >> >> >
> >> >> >> > I am still looking at this scenario as I don't like what I see
> in
> >> >> >> > the
> >> >> >> > service log. I think there might be a bug in the UIMA-AS but I
> am
> >> >> >> > still investigating. Bug or not, what I described above is how
> >> >> >> > UIMA-AS
> >> >> >> > works. You should not expect any replies from a service
> >> >> >> > working on a CAS that was sent through a broker that is bounced
> >> >> >> > before
> >> >> >> > a
> >> >> >> > reply.
> >> >> >> >
> >> >> >> > -jerry
> >> >> >> >
> >> >> >> >
> >> >> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
> >> >> >> > <nelsonrivera12@gmail.com
> >> >> >> >
> >> >> >> > wrote:
> >> >> >> >
> >> >> >> >> My service is a aggregate analysis engine that receives input
> >> >> >> >> CAS
> >> >> >> >> with the direcction of a directory of documents, the first
> >> >> >> >> component
> >> >> >> >> is a MultiplerCas that for each document generate a child CAS,
> >> each
> >> >> >> >> child CAS continues the flow that are other primitive
> annotators
> >> >> >> >> (primitive annotators in c++ that can throw exceptions), then
> >> >> >> >> exists
> >> >> >> >> other MultiplerCas that acts as Merger, waiting the complete
> >> >> >> >> processing of each child CAS and finally generate a child CAS
> >> where
> >> >> >> >> each child CAS previously processed is a view of CAS, this a
> >> >> >> >> child
> >> >> CAS
> >> >> >> >> that is composed of view continúa el flujo a otro AE, que
> espera
> >> un
> >> >> >> >> CAS de esta manera.
> >> >> >> >> I will send you my uima-as descriptor, to understand better my
> >> >> >> >> service. All delegates are co-located.
> >> >> >> >>
> >> >> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how
> >> >> >> >> it
> >> >> >> >> behaves
> >> >> >> >>
> >> >> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
> >:
> >> >> >> >> > I was able to stage this scenario in which an aggregate
> >> >> >> >> > service
> >> >> >> >> > cas
> >> >> >> >> > multiplier receives a single input CAS and
> >> >> >> >> > produces a child CAS.
> >> >> >> >> >
> >> >> >> >> > Scenario:
> >> >> >> >> > 1) Service receives input CAS
> >> >> >> >> > 2) CasMultiplier generates a child CAS
> >> >> >> >> > 3) AE.process() is called (child CAS) and it sleeps for
> awhile
> >> >> >> >> > 4) Broker is shutdown
> >> >> >> >> > 5) AE.process() awakes from sleep
> >> >> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
> >> >> >> >> > 7) Connection is invalid so the child CAS is dropped
> >> >> >> >> > 8) Broker is restarted
> >> >> >> >> > 9) Listeners reconnect:
> >> >> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> >> >> > 10) Broker log shows:
> >> >> >> >> > WARN Service - Async error occurred:
> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> >> >> > destination
> >> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> >> >> exist.
> >> >> >> >> > WARN Service - Async error occurred:
> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> >> >> > destination
> >> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> >> >> exist.
> >> >> >> >> >
> >> >> >> >> > The service failed to send a reply.Nothing is delivered to a
> >> >> >> >> > client
> >> >> >> >> > as
> >> >> >> >> > expected. This is the confusing part since you see a parent
> >> >> >> >> > CAS
> >> >> >> >> > being
> >> >> >> >> sent
> >> >> >> >> > to a client. I just cant imagine how this is possible.
> >> >> >> >> >
> >> >> >> >> > When a service receives an input CAS in a message, it
> extracts
> >> >> >> >> > a
> >> >> JMS
> >> >> >> >> > replyTo object which is a JMS temp queue and stores in an
> >> >> >> >> > internal
> >> >> >> >> > cache
> >> >> >> >> > until it is time to send a reply. Temp queues are transient
> >> >> >> >> > and
> >> >> >> >> > unique
> >> >> >> >> each
> >> >> >> >> > time one is created.
> >> >> >> >> >
> >> >> >> >> > If the broker dies and subsequently restarts, the cached
> >> >> >> >> > replyTo
> >> >> >> >> > queue
> >> >> >> >> > object is invalid (temp queue no longer exist) which leads to
> >> >> >> >> > "Async error occurred:
> >> >> >> >> > org.apache.activemq.DestinationDoesNotExistException" as
> shows
> >> >> >> >> > above.
> >> >> >> >> >
> >> >> >> >> > In the service log I see a bunch of these exceptions:
> >> >> >> >> >
> >> >> >> >> > 02:05:46.574 - 56:
> >> >> >> >> > org.apache.uima.adapter.jms.activemq.
> >> JmsEndpointConnection_impl.
> >> >> >> >> openChannel:
> >> >> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open
> Connection
> >> To
> >> >> >> >> > Broker:
> >> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
> >> >> Silently
> >> >> >> >> > Retrying ...
> >> >> >> >> > 02:05:46.574 - 56:
> >> >> >> >> > org.apache.uima.adapter.jms.activemq.
> >> JmsEndpointConnection_impl.
> >> >> >> >> openChannel:
> >> >> >> >> > WARNING:
> >> >> >> >> > javax.jms.JMSException: Could not connect to broker URL:
> >> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0.
> >> Reason:
> >> >> >> >> > java.net.ConnectException: Connection refused
> >> >> >> >> >
> >> >> >> >> > This is *after* the service is restarted so there seems to be
> >> >> >> >> > a
> >> >> >> problem
> >> >> >> >> > with reestablishing the connection to a new broker.
> >> >> >> >> > I am still investigating that part. I just am confused how
> you
> >> >> >> >> > are
> >> >> >> able
> >> >> >> >> to
> >> >> >> >> > see a reply in the client.
> >> >> >> >> >
> >> >> >> >> > -Jerry
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
> >> >> uimaee@gmail.com>
> >> >> >> >> wrote:
> >> >> >> >> >
> >> >> >> >> >> From what I understand there are two issues:
> >> >> >> >> >>
> >> >> >> >> >> 1) Why service JMS listeners are not reconnecting when a
> >> >> >> >> >> broker
> >> >> >> >> >> is
> >> >> >> >> >> bounced. This
> >> >> >> >> >> should be done by default. The 2.9.0 snapshot may
> address
> >> >> >> >> >> this
> >> >> >> >> >> since
> >> >> >> >> >> a
> >> >> >> >> >> listener
> >> >> >> >> >> uses newer AMQ jars (5.14.0)
> >> >> >> >> >>
> >> >> >> >> >> 2) What happens to a child CAS when a Broker disappears and
> >> >> >> >> >> the
> >> >> >> >> >> service
> >> >> >> >> >> wants
> >> >> >> >> >> to deliver it to a client that is not reachable. I need
> >> >> >> >> >> to
> >> >> >> >> >> investigate
> >> >> >> >> >> this part. I think
> >> >> >> >> >> such CAS is dropped but that should be reflected in a
> log
> >> >> >> >> >> and
> >> >> >> also
> >> >> >> >> >> the
> >> >> >> >> >> input CAS
> >> >> >> >> >> should be marked as failed. Again something for me to
> >> >> >> investigate.
> >> >> >> >> >>
> >> >> >> >> >> I will try to set up a test to see if I can create the
> >> >> >> >> >> scenario.It
> >> >> >> >> >> will
> >> >> >> >> >> take me a bit of time
> >> >> >> >> >> to set this up but I will let you know what happens.
> >> >> >> >> >>
> >> >> >> >> >> -jerry
> >> >> >> >> >>
> >> >> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
> >> >> >> >> >> <uimaee@gmail.com
> >> >> >> >
> >> >> >> >> >> wrote:
> >> >> >> >> >>
> >> >> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
> >> >> >> >> >>> https://repository.apache.org/
> content/repositories/snapshots
> >> >> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-
> 201609
> >> >> >> >> >>> 23.195233-6.tar.gz
> >> >> >> >> >>>
> >> >> >> >> >>> Can you briefly describe your service? Does your service
> >> >> >> >> >>> have
> >> >> >> >> >>> any
> >> >> >> >> remote
> >> >> >> >> >>> delegates or all parts of the pipeline are in the same
> >> process?
> >> >> >> >> >>> Do you see and exceptions in the service log?
> >> >> >> >> >>>
> >> >> >> >> >>> -jerry
> >> >> >> >> >>>
> >> >> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
> >> >> >> >> >>> <ne...@gmail.com>
> >> >> >> >> >>> wrote:
> >> >> >> >> >>>
> >> >> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same
> >> error.
> >> >> >> >> >>>> not
> >> >> >> >> >>>> appears to be broker
> >> >> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
> >> >> >> >> >>>> stagnant
> >> >> >> >> >>>> at
> >> >> >> >> >>>> work
> >> >> >> >> >>>>
> >> >> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <
> >> uimaee@gmail.com
> >> >> >:
> >> >> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is
> >> >> >> >> >>>> > related
> >> >> >> >> >>>> > to
> >> >> >> >> broker
> >> >> >> >> >>>> > issues.
> >> >> >> >> >>>> >
> >> >> >> >> >>>> > -jerry
> >> >> >> >> >>>> >
> >> >> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> >> >> >> >> >>>> nelsonrivera12@gmail.com>
> >> >> >> >> >>>> > wrote:
> >> >> >> >> >>>> >
> >> >> >> >> >>>> >> just like that, the parent CAS comes back but the child
> >> CAS
> >> >> >> >> >>>> >> doesnt.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> And the child CAS if processed, because the uima-as
> >> >> >> >> >>>> >> log
> >> >> >> >> >>>> >> shows
> >> >> >> >> >>>> >> the
> >> >> >> >> >>>> >> lines of the other annotators primitive of the flow
> >> >> processing
> >> >> >> >> >>>> >> the
> >> >> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> uima-as log only shows
> >> >> >> >> >>>> >> "Jms Listener Failed. Endpoint:
> >> >> >> >> >>>> >> temp-queue://ID:localhsot-H81-
> >> M1-33610-1474481197190-1:1:1
> >> >> >> >> >>>> >> Managed
> >> >> >> >> >>>> By:
> >> >> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >> >> >> >>>> >> java.io.EOFException" ........
> >> >> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >> >> >>>> >> XClusterAnalyzerAggregate
> >> >> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
> >> >> >> >> >>>> >> closed..............
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> The CAS is send to service after restart the Broker,
> and
> >> >> >> >> >>>> >> how
> >> >> >> >> >>>> >> say
> >> >> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if
> i
> >> >> >> >> >>>> >> try
> >> >> >> >> quiesce
> >> >> >> >> >>>> >> the Aggregate Service console shows : "Quiescing
> UIMA-AS
> >> >> >> Service.
> >> >> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose
> that
> >> is
> >> >> >> >> >>>> >> the
> >> >> >> >> >>>> >> child
> >> >> >> >> >>>> >> CAS.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> the client if can send more CASes to the service, but
> >> >> >> >> >>>> >> not
> >> >> >> receive
> >> >> >> >> >>>> >> response, for each CAS send after the one CAS that was
> >> send
> >> >> >> after
> >> >> >> >> >>>> >> restart broker (for which parent CAS comes back but
> the
> >> >> child
> >> >> >> >> >>>> >> CAS
> >> >> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> More specific:
> >> >> >> >> >>>> >> scenario 1:
> >> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
> >> >> >> >> >>>> >> Everything
> >> >> >> >> >>>> >> is
> >> >> >> >> >>>> >> fine.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
> >> >> >> >> >>>> >> UIMA-AS
> >> >> >> >> >>>> >> log
> >> >> >> >> >>>> >> show
> >> >> >> >> >>>> >> :
> >> >> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> >> >> parent
> >> >> >> >> >>>> >> CAS
> >> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
> >> >> >> >> >>>> >> client.)
> >> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows :
> >> >> >> "Quiescing
> >> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to
> Process:1"
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> scenario 2:
> >> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
> >> >> >> >> >>>> >> Everything
> >> >> >> >> >>>> >> is
> >> >> >> >> >>>> >> fine.
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
> >> >> >> >> >>>> >> UIMA-AS
> >> >> >> >> >>>> >> log
> >> >> >> >> >>>> >> show
> >> >> >> >> >>>> >> :
> >> >> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> >> >> parent
> >> >> >> >> >>>> >> CAS
> >> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
> >> client)
> >> >> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
> >> >> service.(not
> >> >> >> >> comes
> >> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
> >> >> service.(not
> >> >> >> >> comes
> >> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows:
> >> >> >> >> >>>> >> "Quiescing
> >> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to
> Process:5"
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
> >> >> uimaee@gmail.com
> >> >> >> >:
> >> >> >> >> >>>> >> > When a parent CAS comes back (after broker is
> >> >> >> >> >>>> >> > restored),
> >> >> can
> >> >> >> >> >>>> >> > the
> >> >> >> >> >>>> client
> >> >> >> >> >>>> >> > send more CASes to the service and receive responses?
> >> >> >> >> >>>> >> > I
> >> >> >> >> >>>> >> > am
> >> >> >> >> >>>> >> > trying
> >> >> >> >> >>>> >> > to
> >> >> >> >> >>>> >> > determine what the real problem is here.
> >> >> >> >> >>>> >> >
> >> >> >> >> >>>> >> > 1) JMS Listener not recovering connection
> >> >> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when
> >> broker
> >> >> >> >> >>>> connection is
> >> >> >> >> >>>> >> > stale
> >> >> >> >> >>>> >> >
> >> >> >> >> >>>> >> > -jerry
> >> >> >> >> >>>> >> >
> >> >> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
> >> >> >> >> >>>> uimaee@gmail.com>
> >> >> >> >> >>>> >> > wrote:
> >> >> >> >> >>>> >> >
> >> >> >> >> >>>> >> >> Strange. You say that the parent CAS comes back but
> >> >> >> >> >>>> >> >> the
> >> >> >> >> >>>> >> >> child
> >> >> >> >> CAS
> >> >> >> >> >>>> >> doesnt.
> >> >> >> >> >>>> >> >> Are there any exceptions in the service log? Perhaps
> >> the
> >> >> >> >> >>>> >> >> service
> >> >> >> >> >>>> >> >> is
> >> >> >> >> >>>> >> >> unable
> >> >> >> >> >>>> >> >> to send the child CAS due to missing broker. There
> >> >> >> >> >>>> >> >> should
> >> >> >> >> >>>> >> >> be
> >> >> >> >> >>>> something
> >> >> >> >> >>>> >> in
> >> >> >> >> >>>> >> >> the uima-as log about this.
> >> >> >> >> >>>> >> >>
> >> >> >> >> >>>> >> >> -jerry
> >> >> >> >> >>>> >> >>
> >> >> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >> >> >> >>>> >> >> <ne...@gmail.com>
> >> >> >> >> >>>> >> >> wrote:
> >> >> >> >> >>>> >> >>
> >> >> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
> >> >> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS
> >> >> >> >> >>>> >> >>> continue
> >> >> >> >> >>>> >> >>> in
> >> >> >> >> >>>> >> >>> the
> >> >> >> >> >>>> >> >>> flow,
> >> >> >> >> >>>> >> >>> that are other primitive annotators. And I expected
> >> >> >> >> >>>> >> >>> that
> >> >> >> >> >>>> >> >>> the
> >> >> >> >> >>>> >> >>> child
> >> >> >> >> >>>> >> >>> CAS
> >> >> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>> the scenario:
> >> >> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service
> >> >> >> >> >>>> >> >>> to
> >> >> >> >> >>>> >> >>> verify
> >> >> >> >> >>>> >> >>> everything is fine. The process is correct.
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>> 3- Restart Broker (without restart the service).
> >> >> >> >> >>>> >> >>> UIMA-AS
> >> >> >> log
> >> >> >> >> >>>> show :
> >> >> >> >> >>>> >> >>> "Jms Listener Failed." ....
> >> >> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not
> child
> >> >> >> >> >>>> >> >>> CAS.
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>> Thank for you quick reply
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
> >> >> >> uimaee@gmail.com
> >> >> >> >> >:
> >> >> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
> >> >> >> Multiplier.
> >> >> >> >> >>>> Meaning
> >> >> >> >> >>>> >> >>> given a
> >> >> >> >> >>>> >> >>> > CAS it produces child CASes
> >> >> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this
> >> >> >> >> >>>> >> >>> > what
> >> >> this
> >> >> >> >> >>>> service is
> >> >> >> >> >>>> >> >>> > supposed to do if the broker is running?
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> > Can you confirm the scenario:
> >> >> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate
> service
> >> >> >> >> >>>> >> >>> > 2) Service receives the CAS
> >> >> >> >> >>>> >> >>> > 3) Broker dies
> >> >> >> >> >>>> >> >>> > 4) Broker is restarted
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a
> >> >> >> >> >>>> >> >>> > reply
> >> >> >> >> >>>> >> >>> > when
> >> >> >> >> a
> >> >> >> >> >>>> >> >>> > broker
> >> >> >> >> >>>> >> >>> > is
> >> >> >> >> >>>> >> >>> > restarted?
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> > -Jerry
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
> >> >> >> >> >>>> >> >>> > wrote:
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>> >> I will try the variants that you say. I was
> >> >> >> >> >>>> >> >>> >> deepening
> >> >> >> >> >>>> >> >>> >> the
> >> >> >> >> >>>> problem:
> >> >> >> >> >>>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> I have a Aggregate AE that contains
> >> >> >> >> >>>> >> >>> >> CasMultiplier,
> >> >> >> >> >>>> >> >>> >> the
> >> >> >> >> >>>> >> >>> >> original
> >> >> >> >> >>>> >> >>> >> CAS
> >> >> >> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier,
> >> >> >> >> >>>> >> >>> >> this
> >> >> >> >> >>>> generates
> >> >> >> >> >>>> >> >>> >> new
> >> >> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
> >> >> >> >> >>>> >> >>> >> Aggregate.
> >> >> >> >> >>>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> After restart broker (without restart the
> service
> >> of
> >> >> >> >> >>>> >> >>> >> annotator)
> >> >> >> >> >>>> >> >>> >> and
> >> >> >> >> >>>> >> >>> >> send a CAS to process, the listener associated
> >> >> >> >> >>>> >> >>> >> to
> >> >> >> >> >>>> >> >>> >> the
> >> >> >> >> client
> >> >> >> >> >>>> api
> >> >> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is
> >> >> >> >> >>>> >> >>> >> notified
> >> >> >> >> >>>> >> >>> >> when
> >> >> >> >> >>>> the
> >> >> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas
> generated
> >> in
> >> >> >> >> >>>> >> >>> >> the
> >> >> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when
> finalizes.
> >> >> >> >> >>>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same,
> but
> >> >> >> >> >>>> >> >>> >> how
> >> >> >> >> >>>> >> >>> >> the
> >> >> >> >> >>>> original
> >> >> >> >> >>>> >> >>> >> CAS
> >> >> >> >> >>>> >> >>> >> is notifies into the listener when finalizes,
> >> >> >> >> >>>> >> >>> >> not
> >> >> have
> >> >> >> >> >>>> >> >>> >> problem
> >> >> >> >> >>>> >> >>> >> because is not expected for another new CAS.
> >> >> >> >> >>>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
> >> >> >> >> >>>> >> >>> >> <uimaee@gmail.com
> >> >> >> >> >>>> >:
> >> >> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0
> >> >> >> >> >>>> >> >>> >> > (latest)
> >> >> >> >> >>>> >> >>> >> > and
> >> >> >> use
> >> >> >> >> >>>> >> >>> >> > that
> >> >> >> >> >>>> >> >>> >> > broker
> >> >> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships
> >> >> >> >> >>>> >> >>> >> > with?
> >> >> >> >> >>>> >> >>> >> > There
> >> >> >> >> >>>> >> >>> >> > was
> >> >> >> >> >>>> >> >>> >> > a
> >> >> >> >> >>>> >> >>> >> > nasty
> >> >> >> >> >>>> >> >>> bug
> >> >> >> >> >>>> >> >>> >> > in
> >> >> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not
> working.
> >> >> >> >> >>>> >> >>> >> > I
> >> >> >> >> >>>> >> >>> >> > am
> >> >> >> not
> >> >> >> >> >>>> sure if
> >> >> >> >> >>>> >> >>> >> > this
> >> >> >> >> >>>> >> >>> >> > is
> >> >> >> >> >>>> >> >>> >> > the cause but worth trying.
> >> >> >> >> >>>> >> >>> >> >
> >> >> >> >> >>>> >> >>> >> > You can download AMQ from here:
> >> >> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
> >> >> >> >> >>>> >> >>> >> >
> >> >> >> >> >>>> >> >>> >> > -Jerry
> >> >> >> >> >>>> >> >>> >> >
> >> >> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw
> >> >> >> >> >>>> >> >>> >> > Cwiklik
> >> >> >> >> >>>> >> >>> >> > <ui...@gmail.com>
> >> >> >> >> >>>> >> >>> >> wrote:
> >> >> >> >> >>>> >> >>> >> >
> >> >> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to
> stage
> >> >> >> >> >>>> >> >>> >> >> this
> >> >> >> >> >>>> >> >>> >> >> with
> >> >> >> >> >>>> >> >>> >> >> UIMA-AS
> >> >> >> >> >>>> >> >>> >> >> 2.8.1.
> >> >> >> >> >>>> >> >>> >> I
> >> >> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest)
> code.
> >> >> >> >> >>>> >> >>> >> >> The
> >> >> >> most
> >> >> >> >> >>>> current
> >> >> >> >> >>>> >> >>> >> >> code
> >> >> >> >> >>>> >> >>> >> does
> >> >> >> >> >>>> >> >>> >> >> have newer broker.
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> After the service started, I stopped the
> >> >> >> >> >>>> >> >>> >> >> broker,
> >> >> >> waited
> >> >> >> >> >>>> >> >>> >> >> a
> >> >> >> >> >>>> bit,
> >> >> >> >> >>>> >> and
> >> >> >> >> >>>> >> >>> >> >> restarted the broker.
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following
> in
> >> >> >> >> >>>> >> >>> >> >> the
> >> >> >> log:
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1:
> org.apache.uima.adapter.jms.
> >> >> >> activemq.
> >> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
> >> >> >> >> >>>> >> >>> >> >> INFO:
> >> >> >> >> >>>> >> >>> >> >> Controller:
> >> >> >> >> >>>> >> >>> >> >> Meeting
> >> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
> >> >> >> >> >>>> >> >>> >> >> Endpoint:
> >> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002
> Broker:
> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1:
> org.apache.uima.adapter.jms.
> >> >> >> activemq.
> >> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
> >> >> >> >> >>>> >> >>> >> >> INFO:
> >> >> >> >> >>>> >> >>> >> >> Controller:
> >> >> >> >> >>>> >> >>> >> >> Meeting
> >> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
> >> >> >> >> >>>> >> >>> >> >> Endpoint:
> >> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
> >> >> tcp://localhost:61616
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> >> >> >> WARNING:
> >> >> >> >> >>>> >> Service:
> >> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> >> >> >> WARNING:
> >> >> >> >> >>>> Jms
> >> >> >> >> >>>> >> >>> Listener
> >> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
> >> Managed
> >> >> >> >> >>>> >> >>> >> >> By:
> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> >> >> java.io.EOFException
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> Unable
> >> >> >> To
> >> >> >> >> >>>> Connect
> >> >> >> >> >>>> >> To
> >> >> >> >> >>>> >> >>> >> >> Broker:
> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
> >> >> >> >> >>>> >> >>> >> >> Successful
> >> >> >> >> >>>> >> >>> >> >> ...
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> >> >> >> WARNING:
> >> >> >> >> >>>> >> Service:
> >> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> >> >> >> WARNING:
> >> >> >> >> >>>> Jms
> >> >> >> >> >>>> >> >>> Listener
> >> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
> >> Managed
> >> >> >> >> >>>> >> >>> >> >> By:
> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> >> >> java.io.EOFException
> >> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> Unable
> >> >> >> To
> >> >> >> >> >>>> Connect
> >> >> >> >> >>>> >> To
> >> >> >> >> >>>> >> >>> >> >> Broker:
> >> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
> >> >> >> >> >>>> >> >>> >> >> Successful
> >> >> >> >> >>>> >> >>> >> >> ...
> >> >> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> >> Recovered
> >> >> >> >> >>>> >> Connection
> >> >> >> >> >>>> >> >>> to
> >> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
> >> >> >> >> >>>> >> >>> >> >> Process
> >> >> >> >> >>>> >> >>> >> >> Again
> >> >> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
> >> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> >> Recovered
> >> >> >> >> >>>> >> Connection
> >> >> >> >> >>>> >> >>> to
> >> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
> >> >> >> >> >>>> >> >>> >> >> Process
> >> >> >> >> >>>> >> >>> >> >> Again
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> -Jerry
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson
> rivera
> >> >> >> >> >>>> >> >>> >> >> <
> >> >> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
> >> >> >> >> >>>> >> >>> >> >> wrote:
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
> >> >> happen?
> >> >> >> >> >>>> >> >>> >> >>>
> >> >> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik
> >> >> >> >> >>>> >> >>> >> >>> <
> >> >> >> >> >>>> uimaee@gmail.com
> >> >> >> >> >>>> >> >:
> >> >> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >> >>>> >> >>> >> >>> > -Jerry
> >> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson
> >> rivera
> >> >> >> >> >>>> >> >>> >> >>> > <
> >> >> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >> >> >> >>>> >> >>> >> >>> > wrote:
> >> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an
> >> >> >> >> >>>> >> >>> >> >>> >> annotator
> >> >> >> >> >>>> >> >>> >> >>> >> as
> >> >> >> >> >>>> >> >>> >> >>> >> service,
> >> >> >> >> >>>> >> >>> >> >>> >> processing
> >> >> >> >> >>>> >> >>> >> >>> >> is
> >> >> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker
> >> >> >> >> >>>> >> >>> >> >>> >> is
> >> >> >> stopped
> >> >> >> >> >>>> >> >>> >> >>> >> and
> >> >> >> >> >>>> >> >>> >> >>> >> subsequently
> >> >> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines
> >> >> >> >> >>>> >> >>> >> >>> >> below
> >> >> and
> >> >> >> >> after
> >> >> >> >> >>>> this
> >> >> >> >> >>>> >> >>> >> >>> >> the
> >> >> >> >> >>>> >> >>> >> >>> >> api
> >> >> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the
> >> >> >> >> >>>> >> >>> >> >>> >> listener
> >> >> >> when
> >> >> >> >> >>>> >> >>> >> >>> >> a
> >> >> >> >> >>>> cas
> >> >> >> >> >>>> >> >>> >> >>> >> is
> >> >> >> >> >>>> >> >>> >> >>> >> processed, any more.
> >> >> >> >> >>>> >> >>> >> >>> >> can help me?:
> >> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> >> Analyzer
> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >> >>>> >> >>> Exception
> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >> >>>> >> >>> javax.jms.JMSException:
> >> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
> >> >> >> >> >>>> >> >>> >> >>> >> Cluster
> >> >> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >> >> >>>> Retrying
> >> >> >> >> >>>> >> >>> >> >>> >> ...
> >> >> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
> >> >> >> >> >>>> >> >>> >> >>> >> Failed.
> >> >> >> >> >>>> Endpoint:
> >> >> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> >> >> >> >> >>>> M1-33610-1474481197190-1:1:1
> >> >> >> >> >>>> >> >>> Managed
> >> >> >> >> >>>> >> >>> >> >>> >> By:
> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
> >> >> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
> >> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
> >> >> >> >> >>>> >> >>> >> >>> >> Cluster
> >> >> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
> >> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> >> Analyzer
> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >> >>>> >> >>> Exception
> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.
> >> ConnectionClosedException:
> >> >> >> The
> >> >> >> >> >>>> >> connection
> >> >> >> >> >>>> >> >>> is
> >> >> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> >> Analyzer
> >> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >> >>>> >> >>> Exception
> >> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >> >>>> >> >>> nerCont
> >> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
> >> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
> >> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The
> >> Consumer
> >> >> is
> >> >> >> >> >>>> closed.
> >> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >> >>>> >> >>> >> >>>
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >>
> >> >> >> >> >>>> >> >>> >> >
> >> >> >> >> >>>> >> >>> >>
> >> >> >> >> >>>> >> >>> >
> >> >> >> >> >>>> >> >>>
> >> >> >> >> >>>> >> >>
> >> >> >> >> >>>> >> >>
> >> >> >> >> >>>> >> >
> >> >> >> >> >>>> >>
> >> >> >> >> >>>> >
> >> >> >> >> >>>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >>
> >
>
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
I tried the snapshot that you gave me, and I understand perfectly the
explanation of recovery queue that you explain, but to me not work,
and y get the same behavior. With the last snapshop, the uima-as log
never show recovery of new queue after restart broker (restart =
execute /bin/startBroker.sh Starting in foreground, then pressing
CTRL+C, then execute startBroker.sh again), only shows:
08:58:13.825 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint:
ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
08:58:13.825 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
08:58:13.825 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleTempQueueFailure:
WARNING: Jms Listener Failed. Endpoint:
temp-queue://ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
08:58:13.825 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying ...
08:58:13.839 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
08:58:13.844 - 46:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
08:58:13.844 - 46:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying ...
08:58:13.876 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
Established Connection to Broker: tcp://localhost:61616
08:58:16.719 - 46:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
Established Connection to Broker: tcp://localhost:61616
08:59:16.887 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
javax.jms.IllegalStateException: The Consumer is closed
08:59:16.887 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying ...
08:59:16.900 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
Established Connection to Broker: tcp://localhost:61616
08:59:16.917 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
08:59:16.917 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
javax.jms.IllegalStateException: The Consumer is closed
08:59:16.917 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying ...
08:59:16.948 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
Established Connection to Broker: tcp://localhost:61616
09:01:17.93 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
javax.jms.IllegalStateException: The Consumer is closed
09:01:17.93 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying ...
09:01:17.109 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
Established Connection to Broker: tcp://localhost:61616
And in the console output of service:
Could not refresh JMS Connection for destination
'queue://XClusterAnalyzerAggregate' - silently retrying in 5 ms.
Cause: Could not connect to broker URL: tcp://localhost:61616. Reason:
java.net.ConnectException: Connection refused
2016-10-27 14:29 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Nelson, try this fix:
> https://repository.apache.org/content/repositories/snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-20161027.181519-7.tar.gz
>
> Keep in mind that if you bounce a broker while a CAS is being processed in
> a service, the client will not be able to get the input CAS back nor will
> it get child CASes. Both the input and child CAS will be thrown away by the
> service due to undeliverable destination (old temp queue). The code will
> actually stop the Cas Multiplier from producing child CASes if it detects
> an undeliverable destination. You need a process timeout on the client
> side to deal with that. When a broker is bounced, both the client and a
> service will detect a new broker instance, kill its old temp queue
> listeners and create new ones so that subsequent processing can take place.
>
> Jerry
>
> On Thu, Oct 27, 2016 at 1:19 PM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> Hi, I would like know if you have worked in the solution of this
>> theme, related with recovered of queues in service cas multiplier,
>> after restart uima-as broker, specifically temp queue. If you have a
>> snapshot with this. Thanks
>>
>> 2016-09-28 10:34 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Nelson, yes I think there is a problem in the UIMA-AS. First, there is
>> > evidence that a service
>> > is trying to use a stale temp queue:
>> >
>> > tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
>> > Cannot use a Temporary destination from another Connection.
>> >
>> >
>> > Also, I think there is another problem related to recovery of temp
>> > queues
>> > in the service.
>> > The service cas multiplier uses multiple queues:
>> > 1) input queue - receives GetMeta and Process requests from a client
>> > 2) temp queue - receives FreeCas requests from a client. This is only
>> > created when a service
>> > is a Cas Multiplier. This is used to free CASes in a cas
>> > multiplier.
>> >
>> > The input queue listener recovers the connection since it received an
>> input
>> > CAS. I think the
>> > temp queue is not recovered and this leads to repeated retries.
>> >
>> > I will work on fixing this today and let you know when this is ready. I
>> > will also create a new
>> > snapshot for you to try.
>> >
>> > Thanks for finding these problems.
>> >
>> > - jerry
>> >
>> >
>> > On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <
>> nelsonrivera12@gmail.com>
>> > wrote:
>> >
>> >> ok, only show you the result of last test.
>> >> Now in my test with uima-as-2.9.0, after complete restart of the
>> >> Broker UIMA-AS log show
>> >>
>> >> 09:51:01.731 - 19:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.
>> >> handleListenerSetupFailure:
>> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> >> 09:51:01.740 - 46:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.
>> >> handleListenerSetupFailure:
>> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> >> 09:51:01.747 - 15:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.
>> >> handleListenerSetupFailure:
>> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> >> 09:51:01.773 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 09:51:01.773 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 09:51:01.774 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> java.io.EOFException
>> >> 09:51:01.774 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.
>> >> handleListenerSetupFailure:
>> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
>> >> Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
>> >> 09:51:01.777 - 19:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 09:51:01.777 - 19:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> WARNING: Jms Listener Failed. Endpoint:
>> >> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
>> >> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
>> >> Cannot use a Temporary destination from another Connection.
>> >>
>> >> this lines are continue to repeat in the log indefinitely. And after
>> >> this have the same problem when send input Cas at the service
>> >>
>> >> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > Nelson, I will investigate your use case and let you know.
>> >> >
>> >> > -jerry
>> >> >
>> >> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera
>> >> > <nelsonrivera12@gmail.com
>> >> >
>> >> > wrote:
>> >> >
>> >> >> Thank you for describe how UIMA-AS works, it is very helpful. But
>> >> >> my
>> >> >> scenario is not restart the broker while the uima-as service works
>> >> >> on
>> >> >> input CAS
>> >> >>
>> >> >> my scenario is
>> >> >> 1- Start Broker and UIMA-AS aggregate service
>> >> >>
>> >> >> 2- Complete Restart of the Broker (without restart the service).
>> >> >> UIMA-AS log show the lines that previously i said you:
>> >> >> "Jms Listener Failed." ....
>> >> >> "The Session is closed." ...
>> >> >> "The Consumer is closed." ...
>> >> >>
>> >> >> I do not see in my uima-as log the lines that you say in your test
>> >> >> after broker is bounced:
>> >> >> Listeners reconnect:
>> >> >> INFO | Successfully refreshed JMS Connection
>> >> >> INFO | Successfully refreshed JMS Connection
>> >> >> In my case (uima-as 2.8.1) seems to be a problem with
>> >> >> reestablishing
>> >> >> the connection to the broker after restart.
>> >> >>
>> >> >> 4- Then in this point, after the broker was restarted completely,
>> >> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for
>> >> >> this
>> >> >> petition UIMA-AS client receive the parent CAS, but not child CAS.
>> >> >>
>> >> >> In this moment i will trys with uima-as-2.9.0 snapshot
>> >> >>
>> >> >>
>> >> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >> > Nelson, in my scenario I do have a Cas Multiplier which produces
>> >> >> > a
>> >> >> > child
>> >> >> > CAS for each input CAS. For the purpose of this
>> >> >> > test its not important how many internal delegates there are in a
>> >> >> pipeline.
>> >> >> > The service is configured as a Cas Multiplier
>> >> >> > and it does send child CASes when a broker is running.
>> >> >> >
>> >> >> > The problem is that once the input CAS is received, its reply
>> >> >> > queue
>> >> >> > is
>> >> >> > bound to a specific temp reply queue. When a broker is
>> >> >> > bounced, that queue is not valid anymore and cant be used to
>> >> >> > return
>> >> >> > child
>> >> >> > CASes nor the input CAS. To guard against such
>> >> >> > cases you need to use timeouts on the client side.
>> >> >> >
>> >> >> > Like I said in my previous email I dont get how an input CAS can
>> >> >> > be
>> >> >> > returned back to the client after a broker bounce. This should
>> >> >> > not happen since its temp queue does not exist in a new broker.
>> >> >> >
>> >> >> > I am still looking at this scenario as I don't like what I see in
>> >> >> > the
>> >> >> > service log. I think there might be a bug in the UIMA-AS but I am
>> >> >> > still investigating. Bug or not, what I described above is how
>> >> >> > UIMA-AS
>> >> >> > works. You should not expect any replies from a service
>> >> >> > working on a CAS that was sent through a broker that is bounced
>> >> >> > before
>> >> >> > a
>> >> >> > reply.
>> >> >> >
>> >> >> > -jerry
>> >> >> >
>> >> >> >
>> >> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
>> >> >> > <nelsonrivera12@gmail.com
>> >> >> >
>> >> >> > wrote:
>> >> >> >
>> >> >> >> My service is a aggregate analysis engine that receives input
>> >> >> >> CAS
>> >> >> >> with the direcction of a directory of documents, the first
>> >> >> >> component
>> >> >> >> is a MultiplerCas that for each document generate a child CAS,
>> each
>> >> >> >> child CAS continues the flow that are other primitive annotators
>> >> >> >> (primitive annotators in c++ that can throw exceptions), then
>> >> >> >> exists
>> >> >> >> other MultiplerCas that acts as Merger, waiting the complete
>> >> >> >> processing of each child CAS and finally generate a child CAS
>> where
>> >> >> >> each child CAS previously processed is a view of CAS, this a
>> >> >> >> child
>> >> CAS
>> >> >> >> that is composed of view continúa el flujo a otro AE, que espera
>> un
>> >> >> >> CAS de esta manera.
>> >> >> >> I will send you my uima-as descriptor, to understand better my
>> >> >> >> service. All delegates are co-located.
>> >> >> >>
>> >> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how
>> >> >> >> it
>> >> >> >> behaves
>> >> >> >>
>> >> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >> >> > I was able to stage this scenario in which an aggregate
>> >> >> >> > service
>> >> >> >> > cas
>> >> >> >> > multiplier receives a single input CAS and
>> >> >> >> > produces a child CAS.
>> >> >> >> >
>> >> >> >> > Scenario:
>> >> >> >> > 1) Service receives input CAS
>> >> >> >> > 2) CasMultiplier generates a child CAS
>> >> >> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
>> >> >> >> > 4) Broker is shutdown
>> >> >> >> > 5) AE.process() awakes from sleep
>> >> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
>> >> >> >> > 7) Connection is invalid so the child CAS is dropped
>> >> >> >> > 8) Broker is restarted
>> >> >> >> > 9) Listeners reconnect:
>> >> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> >> > 10) Broker log shows:
>> >> >> >> > WARN Service - Async error occurred:
>> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> >> > destination
>> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
>> >> exist.
>> >> >> >> > WARN Service - Async error occurred:
>> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> >> > destination
>> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
>> >> exist.
>> >> >> >> >
>> >> >> >> > The service failed to send a reply.Nothing is delivered to a
>> >> >> >> > client
>> >> >> >> > as
>> >> >> >> > expected. This is the confusing part since you see a parent
>> >> >> >> > CAS
>> >> >> >> > being
>> >> >> >> sent
>> >> >> >> > to a client. I just cant imagine how this is possible.
>> >> >> >> >
>> >> >> >> > When a service receives an input CAS in a message, it extracts
>> >> >> >> > a
>> >> JMS
>> >> >> >> > replyTo object which is a JMS temp queue and stores in an
>> >> >> >> > internal
>> >> >> >> > cache
>> >> >> >> > until it is time to send a reply. Temp queues are transient
>> >> >> >> > and
>> >> >> >> > unique
>> >> >> >> each
>> >> >> >> > time one is created.
>> >> >> >> >
>> >> >> >> > If the broker dies and subsequently restarts, the cached
>> >> >> >> > replyTo
>> >> >> >> > queue
>> >> >> >> > object is invalid (temp queue no longer exist) which leads to
>> >> >> >> > "Async error occurred:
>> >> >> >> > org.apache.activemq.DestinationDoesNotExistException" as shows
>> >> >> >> > above.
>> >> >> >> >
>> >> >> >> > In the service log I see a bunch of these exceptions:
>> >> >> >> >
>> >> >> >> > 02:05:46.574 - 56:
>> >> >> >> > org.apache.uima.adapter.jms.activemq.
>> JmsEndpointConnection_impl.
>> >> >> >> openChannel:
>> >> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection
>> To
>> >> >> >> > Broker:
>> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
>> >> Silently
>> >> >> >> > Retrying ...
>> >> >> >> > 02:05:46.574 - 56:
>> >> >> >> > org.apache.uima.adapter.jms.activemq.
>> JmsEndpointConnection_impl.
>> >> >> >> openChannel:
>> >> >> >> > WARNING:
>> >> >> >> > javax.jms.JMSException: Could not connect to broker URL:
>> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0.
>> Reason:
>> >> >> >> > java.net.ConnectException: Connection refused
>> >> >> >> >
>> >> >> >> > This is *after* the service is restarted so there seems to be
>> >> >> >> > a
>> >> >> problem
>> >> >> >> > with reestablishing the connection to a new broker.
>> >> >> >> > I am still investigating that part. I just am confused how you
>> >> >> >> > are
>> >> >> able
>> >> >> >> to
>> >> >> >> > see a reply in the client.
>> >> >> >> >
>> >> >> >> > -Jerry
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
>> >> uimaee@gmail.com>
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> From what I understand there are two issues:
>> >> >> >> >>
>> >> >> >> >> 1) Why service JMS listeners are not reconnecting when a
>> >> >> >> >> broker
>> >> >> >> >> is
>> >> >> >> >> bounced. This
>> >> >> >> >> should be done by default. The 2.9.0 snapshot may address
>> >> >> >> >> this
>> >> >> >> >> since
>> >> >> >> >> a
>> >> >> >> >> listener
>> >> >> >> >> uses newer AMQ jars (5.14.0)
>> >> >> >> >>
>> >> >> >> >> 2) What happens to a child CAS when a Broker disappears and
>> >> >> >> >> the
>> >> >> >> >> service
>> >> >> >> >> wants
>> >> >> >> >> to deliver it to a client that is not reachable. I need
>> >> >> >> >> to
>> >> >> >> >> investigate
>> >> >> >> >> this part. I think
>> >> >> >> >> such CAS is dropped but that should be reflected in a log
>> >> >> >> >> and
>> >> >> also
>> >> >> >> >> the
>> >> >> >> >> input CAS
>> >> >> >> >> should be marked as failed. Again something for me to
>> >> >> investigate.
>> >> >> >> >>
>> >> >> >> >> I will try to set up a test to see if I can create the
>> >> >> >> >> scenario.It
>> >> >> >> >> will
>> >> >> >> >> take me a bit of time
>> >> >> >> >> to set this up but I will let you know what happens.
>> >> >> >> >>
>> >> >> >> >> -jerry
>> >> >> >> >>
>> >> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
>> >> >> >> >> <uimaee@gmail.com
>> >> >> >
>> >> >> >> >> wrote:
>> >> >> >> >>
>> >> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> >> >> >> >>> https://repository.apache.org/content/repositories/snapshots
>> >> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> >> >> >> >>> 23.195233-6.tar.gz
>> >> >> >> >>>
>> >> >> >> >>> Can you briefly describe your service? Does your service
>> >> >> >> >>> have
>> >> >> >> >>> any
>> >> >> >> remote
>> >> >> >> >>> delegates or all parts of the pipeline are in the same
>> process?
>> >> >> >> >>> Do you see and exceptions in the service log?
>> >> >> >> >>>
>> >> >> >> >>> -jerry
>> >> >> >> >>>
>> >> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>> >> >> >> >>> <ne...@gmail.com>
>> >> >> >> >>> wrote:
>> >> >> >> >>>
>> >> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same
>> error.
>> >> >> >> >>>> not
>> >> >> >> >>>> appears to be broker
>> >> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
>> >> >> >> >>>> stagnant
>> >> >> >> >>>> at
>> >> >> >> >>>> work
>> >> >> >> >>>>
>> >> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <
>> uimaee@gmail.com
>> >> >:
>> >> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is
>> >> >> >> >>>> > related
>> >> >> >> >>>> > to
>> >> >> >> broker
>> >> >> >> >>>> > issues.
>> >> >> >> >>>> >
>> >> >> >> >>>> > -jerry
>> >> >> >> >>>> >
>> >> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> >> >> >> >>>> nelsonrivera12@gmail.com>
>> >> >> >> >>>> > wrote:
>> >> >> >> >>>> >
>> >> >> >> >>>> >> just like that, the parent CAS comes back but the child
>> CAS
>> >> >> >> >>>> >> doesnt.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> And the child CAS if processed, because the uima-as
>> >> >> >> >>>> >> log
>> >> >> >> >>>> >> shows
>> >> >> >> >>>> >> the
>> >> >> >> >>>> >> lines of the other annotators primitive of the flow
>> >> processing
>> >> >> >> >>>> >> the
>> >> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> uima-as log only shows
>> >> >> >> >>>> >> "Jms Listener Failed. Endpoint:
>> >> >> >> >>>> >> temp-queue://ID:localhsot-H81-
>> M1-33610-1474481197190-1:1:1
>> >> >> >> >>>> >> Managed
>> >> >> >> >>>> By:
>> >> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >> >> >>>> >> java.io.EOFException" ........
>> >> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >> >>>> >> XClusterAnalyzerAggregate
>> >> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
>> >> >> >> >>>> >> closed..............
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> The CAS is send to service after restart the Broker, and
>> >> >> >> >>>> >> how
>> >> >> >> >>>> >> say
>> >> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i
>> >> >> >> >>>> >> try
>> >> >> >> quiesce
>> >> >> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
>> >> >> Service.
>> >> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that
>> is
>> >> >> >> >>>> >> the
>> >> >> >> >>>> >> child
>> >> >> >> >>>> >> CAS.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> the client if can send more CASes to the service, but
>> >> >> >> >>>> >> not
>> >> >> receive
>> >> >> >> >>>> >> response, for each CAS send after the one CAS that was
>> send
>> >> >> after
>> >> >> >> >>>> >> restart broker (for which parent CAS comes back but the
>> >> child
>> >> >> >> >>>> >> CAS
>> >> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> More specific:
>> >> >> >> >>>> >> scenario 1:
>> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
>> >> >> >> >>>> >> Everything
>> >> >> >> >>>> >> is
>> >> >> >> >>>> >> fine.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
>> >> >> >> >>>> >> UIMA-AS
>> >> >> >> >>>> >> log
>> >> >> >> >>>> >> show
>> >> >> >> >>>> >> :
>> >> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
>> >> parent
>> >> >> >> >>>> >> CAS
>> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
>> >> >> >> >>>> >> client.)
>> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows :
>> >> >> "Quiescing
>> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> scenario 2:
>> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service.
>> >> >> >> >>>> >> Everything
>> >> >> >> >>>> >> is
>> >> >> >> >>>> >> fine.
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> 2- Restart Broker (without restart the service).
>> >> >> >> >>>> >> UIMA-AS
>> >> >> >> >>>> >> log
>> >> >> >> >>>> >> show
>> >> >> >> >>>> >> :
>> >> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
>> >> parent
>> >> >> >> >>>> >> CAS
>> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
>> client)
>> >> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
>> >> service.(not
>> >> >> >> comes
>> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
>> >> service.(not
>> >> >> >> comes
>> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows:
>> >> >> >> >>>> >> "Quiescing
>> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>> >> >> >> >>>> >>
>> >> >> >> >>>> >>
>> >> >> >> >>>> >>
>> >> >> >> >>>> >>
>> >> >> >> >>>> >>
>> >> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
>> >> uimaee@gmail.com
>> >> >> >:
>> >> >> >> >>>> >> > When a parent CAS comes back (after broker is
>> >> >> >> >>>> >> > restored),
>> >> can
>> >> >> >> >>>> >> > the
>> >> >> >> >>>> client
>> >> >> >> >>>> >> > send more CASes to the service and receive responses?
>> >> >> >> >>>> >> > I
>> >> >> >> >>>> >> > am
>> >> >> >> >>>> >> > trying
>> >> >> >> >>>> >> > to
>> >> >> >> >>>> >> > determine what the real problem is here.
>> >> >> >> >>>> >> >
>> >> >> >> >>>> >> > 1) JMS Listener not recovering connection
>> >> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when
>> broker
>> >> >> >> >>>> connection is
>> >> >> >> >>>> >> > stale
>> >> >> >> >>>> >> >
>> >> >> >> >>>> >> > -jerry
>> >> >> >> >>>> >> >
>> >> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>> >> >> >> >>>> uimaee@gmail.com>
>> >> >> >> >>>> >> > wrote:
>> >> >> >> >>>> >> >
>> >> >> >> >>>> >> >> Strange. You say that the parent CAS comes back but
>> >> >> >> >>>> >> >> the
>> >> >> >> >>>> >> >> child
>> >> >> >> CAS
>> >> >> >> >>>> >> doesnt.
>> >> >> >> >>>> >> >> Are there any exceptions in the service log? Perhaps
>> the
>> >> >> >> >>>> >> >> service
>> >> >> >> >>>> >> >> is
>> >> >> >> >>>> >> >> unable
>> >> >> >> >>>> >> >> to send the child CAS due to missing broker. There
>> >> >> >> >>>> >> >> should
>> >> >> >> >>>> >> >> be
>> >> >> >> >>>> something
>> >> >> >> >>>> >> in
>> >> >> >> >>>> >> >> the uima-as log about this.
>> >> >> >> >>>> >> >>
>> >> >> >> >>>> >> >> -jerry
>> >> >> >> >>>> >> >>
>> >> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> >> >> >>>> >> >> <ne...@gmail.com>
>> >> >> >> >>>> >> >> wrote:
>> >> >> >> >>>> >> >>
>> >> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
>> >> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS
>> >> >> >> >>>> >> >>> continue
>> >> >> >> >>>> >> >>> in
>> >> >> >> >>>> >> >>> the
>> >> >> >> >>>> >> >>> flow,
>> >> >> >> >>>> >> >>> that are other primitive annotators. And I expected
>> >> >> >> >>>> >> >>> that
>> >> >> >> >>>> >> >>> the
>> >> >> >> >>>> >> >>> child
>> >> >> >> >>>> >> >>> CAS
>> >> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>> the scenario:
>> >> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service
>> >> >> >> >>>> >> >>> to
>> >> >> >> >>>> >> >>> verify
>> >> >> >> >>>> >> >>> everything is fine. The process is correct.
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>> 3- Restart Broker (without restart the service).
>> >> >> >> >>>> >> >>> UIMA-AS
>> >> >> log
>> >> >> >> >>>> show :
>> >> >> >> >>>> >> >>> "Jms Listener Failed." ....
>> >> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child
>> >> >> >> >>>> >> >>> CAS.
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>> Thank for you quick reply
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
>> >> >> uimaee@gmail.com
>> >> >> >> >:
>> >> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
>> >> >> Multiplier.
>> >> >> >> >>>> Meaning
>> >> >> >> >>>> >> >>> given a
>> >> >> >> >>>> >> >>> > CAS it produces child CASes
>> >> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this
>> >> >> >> >>>> >> >>> > what
>> >> this
>> >> >> >> >>>> service is
>> >> >> >> >>>> >> >>> > supposed to do if the broker is running?
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> > Can you confirm the scenario:
>> >> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >> >> >> >>>> >> >>> > 2) Service receives the CAS
>> >> >> >> >>>> >> >>> > 3) Broker dies
>> >> >> >> >>>> >> >>> > 4) Broker is restarted
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a
>> >> >> >> >>>> >> >>> > reply
>> >> >> >> >>>> >> >>> > when
>> >> >> >> a
>> >> >> >> >>>> >> >>> > broker
>> >> >> >> >>>> >> >>> > is
>> >> >> >> >>>> >> >>> > restarted?
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> > -Jerry
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >> >>>> >> >>> > wrote:
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>> >> I will try the variants that you say. I was
>> >> >> >> >>>> >> >>> >> deepening
>> >> >> >> >>>> >> >>> >> the
>> >> >> >> >>>> problem:
>> >> >> >> >>>> >> >>> >>
>> >> >> >> >>>> >> >>> >> I have a Aggregate AE that contains
>> >> >> >> >>>> >> >>> >> CasMultiplier,
>> >> >> >> >>>> >> >>> >> the
>> >> >> >> >>>> >> >>> >> original
>> >> >> >> >>>> >> >>> >> CAS
>> >> >> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier,
>> >> >> >> >>>> >> >>> >> this
>> >> >> >> >>>> generates
>> >> >> >> >>>> >> >>> >> new
>> >> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
>> >> >> >> >>>> >> >>> >> Aggregate.
>> >> >> >> >>>> >> >>> >>
>> >> >> >> >>>> >> >>> >> After restart broker (without restart the service
>> of
>> >> >> >> >>>> >> >>> >> annotator)
>> >> >> >> >>>> >> >>> >> and
>> >> >> >> >>>> >> >>> >> send a CAS to process, the listener associated
>> >> >> >> >>>> >> >>> >> to
>> >> >> >> >>>> >> >>> >> the
>> >> >> >> client
>> >> >> >> >>>> api
>> >> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is
>> >> >> >> >>>> >> >>> >> notified
>> >> >> >> >>>> >> >>> >> when
>> >> >> >> >>>> the
>> >> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated
>> in
>> >> >> >> >>>> >> >>> >> the
>> >> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >> >> >> >>>> >> >>> >>
>> >> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but
>> >> >> >> >>>> >> >>> >> how
>> >> >> >> >>>> >> >>> >> the
>> >> >> >> >>>> original
>> >> >> >> >>>> >> >>> >> CAS
>> >> >> >> >>>> >> >>> >> is notifies into the listener when finalizes,
>> >> >> >> >>>> >> >>> >> not
>> >> have
>> >> >> >> >>>> >> >>> >> problem
>> >> >> >> >>>> >> >>> >> because is not expected for another new CAS.
>> >> >> >> >>>> >> >>> >>
>> >> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>> >> >> >> >>>> >> >>> >> <uimaee@gmail.com
>> >> >> >> >>>> >:
>> >> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0
>> >> >> >> >>>> >> >>> >> > (latest)
>> >> >> >> >>>> >> >>> >> > and
>> >> >> use
>> >> >> >> >>>> >> >>> >> > that
>> >> >> >> >>>> >> >>> >> > broker
>> >> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships
>> >> >> >> >>>> >> >>> >> > with?
>> >> >> >> >>>> >> >>> >> > There
>> >> >> >> >>>> >> >>> >> > was
>> >> >> >> >>>> >> >>> >> > a
>> >> >> >> >>>> >> >>> >> > nasty
>> >> >> >> >>>> >> >>> bug
>> >> >> >> >>>> >> >>> >> > in
>> >> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working.
>> >> >> >> >>>> >> >>> >> > I
>> >> >> >> >>>> >> >>> >> > am
>> >> >> not
>> >> >> >> >>>> sure if
>> >> >> >> >>>> >> >>> >> > this
>> >> >> >> >>>> >> >>> >> > is
>> >> >> >> >>>> >> >>> >> > the cause but worth trying.
>> >> >> >> >>>> >> >>> >> >
>> >> >> >> >>>> >> >>> >> > You can download AMQ from here:
>> >> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
>> >> >> >> >>>> >> >>> >> >
>> >> >> >> >>>> >> >>> >> > -Jerry
>> >> >> >> >>>> >> >>> >> >
>> >> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw
>> >> >> >> >>>> >> >>> >> > Cwiklik
>> >> >> >> >>>> >> >>> >> > <ui...@gmail.com>
>> >> >> >> >>>> >> >>> >> wrote:
>> >> >> >> >>>> >> >>> >> >
>> >> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage
>> >> >> >> >>>> >> >>> >> >> this
>> >> >> >> >>>> >> >>> >> >> with
>> >> >> >> >>>> >> >>> >> >> UIMA-AS
>> >> >> >> >>>> >> >>> >> >> 2.8.1.
>> >> >> >> >>>> >> >>> >> I
>> >> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code.
>> >> >> >> >>>> >> >>> >> >> The
>> >> >> most
>> >> >> >> >>>> current
>> >> >> >> >>>> >> >>> >> >> code
>> >> >> >> >>>> >> >>> >> does
>> >> >> >> >>>> >> >>> >> >> have newer broker.
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> After the service started, I stopped the
>> >> >> >> >>>> >> >>> >> >> broker,
>> >> >> waited
>> >> >> >> >>>> >> >>> >> >> a
>> >> >> >> >>>> bit,
>> >> >> >> >>>> >> and
>> >> >> >> >>>> >> >>> >> >> restarted the broker.
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in
>> >> >> >> >>>> >> >>> >> >> the
>> >> >> log:
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
>> >> >> activemq.
>> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
>> >> >> >> >>>> >> >>> >> >> INFO:
>> >> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
>> >> >> activemq.
>> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners:
>> >> >> >> >>>> >> >>> >> >> INFO:
>> >> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
>> >> tcp://localhost:61616
>> >> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> ainer.onException:
>> >> >> >> WARNING:
>> >> >> >> >>>> >> Service:
>> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> ainer.onException:
>> >> >> >> WARNING:
>> >> >> >> >>>> Jms
>> >> >> >> >>>> >> >>> Listener
>> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
>> Managed
>> >> >> >> >>>> >> >>> >> >> By:
>> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> >> java.io.EOFException
>> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> Unable
>> >> >> To
>> >> >> >> >>>> Connect
>> >> >> >> >>>> >> To
>> >> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
>> >> >> >> >>>> >> >>> >> >> Successful
>> >> >> >> >>>> >> >>> >> >> ...
>> >> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> ainer.onException:
>> >> >> >> WARNING:
>> >> >> >> >>>> >> Service:
>> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> ainer.onException:
>> >> >> >> WARNING:
>> >> >> >> >>>> Jms
>> >> >> >> >>>> >> >>> Listener
>> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
>> Managed
>> >> >> >> >>>> >> >>> >> >> By:
>> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> >> java.io.EOFException
>> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> Unable
>> >> >> To
>> >> >> >> >>>> Connect
>> >> >> >> >>>> >> To
>> >> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until
>> >> >> >> >>>> >> >>> >> >> Successful
>> >> >> >> >>>> >> >>> >> >> ...
>> >> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> >> Recovered
>> >> >> >> >>>> >> Connection
>> >> >> >> >>>> >> >>> to
>> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
>> >> >> >> >>>> >> >>> >> >> Process
>> >> >> >> >>>> >> >>> >> >> Again
>> >> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
>> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> WARNING:
>> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> >> Recovered
>> >> >> >> >>>> >> Connection
>> >> >> >> >>>> >> >>> to
>> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to
>> >> >> >> >>>> >> >>> >> >> Process
>> >> >> >> >>>> >> >>> >> >> Again
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> -Jerry
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera
>> >> >> >> >>>> >> >>> >> >> <
>> >> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
>> >> >> >> >>>> >> >>> >> >> wrote:
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
>> >> happen?
>> >> >> >> >>>> >> >>> >> >>>
>> >> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik
>> >> >> >> >>>> >> >>> >> >>> <
>> >> >> >> >>>> uimaee@gmail.com
>> >> >> >> >>>> >> >:
>> >> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >>>> >> >>> >> >>> > -Jerry
>> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson
>> rivera
>> >> >> >> >>>> >> >>> >> >>> > <
>> >> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >> >>>> >> >>> >> >>> > wrote:
>> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an
>> >> >> >> >>>> >> >>> >> >>> >> annotator
>> >> >> >> >>>> >> >>> >> >>> >> as
>> >> >> >> >>>> >> >>> >> >>> >> service,
>> >> >> >> >>>> >> >>> >> >>> >> processing
>> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker
>> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> stopped
>> >> >> >> >>>> >> >>> >> >>> >> and
>> >> >> >> >>>> >> >>> >> >>> >> subsequently
>> >> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines
>> >> >> >> >>>> >> >>> >> >>> >> below
>> >> and
>> >> >> >> after
>> >> >> >> >>>> this
>> >> >> >> >>>> >> >>> >> >>> >> the
>> >> >> >> >>>> >> >>> >> >>> >> api
>> >> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the
>> >> >> >> >>>> >> >>> >> >>> >> listener
>> >> >> when
>> >> >> >> >>>> >> >>> >> >>> >> a
>> >> >> >> >>>> cas
>> >> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >> >>>> >> >>> >> >>> >> processed, any more.
>> >> >> >> >>>> >> >>> >> >>> >> can help me?:
>> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> Analyzer
>> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >>>> >> >>> Exception
>> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >> >>>> >> >>> javax.jms.JMSException:
>> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
>> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >> >>>> Retrying
>> >> >> >> >>>> >> >>> >> >>> >> ...
>> >> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
>> >> >> >> >>>> >> >>> >> >>> >> Failed.
>> >> >> >> >>>> Endpoint:
>> >> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> >> >> >> >>>> M1-33610-1474481197190-1:1:1
>> >> >> >> >>>> >> >>> Managed
>> >> >> >> >>>> >> >>> >> >>> >> By:
>> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
>> >> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
>> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
>> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> Analyzer
>> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >>>> >> >>> Exception
>> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.
>> ConnectionClosedException:
>> >> >> The
>> >> >> >> >>>> >> connection
>> >> >> >> >>>> >> >>> is
>> >> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> >> Analyzer
>> >> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >> >>>> >> >>> Exception
>> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >> >>>> >> >>> nerCont
>> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
>> >> >> >> >>>> >> >>> >> >>> >> Endpoint:
>> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The
>> Consumer
>> >> is
>> >> >> >> >>>> closed.
>> >> >> >> >>>> >> >>> >> >>> >>
>> >> >> >> >>>> >> >>> >> >>> >
>> >> >> >> >>>> >> >>> >> >>>
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >>
>> >> >> >> >>>> >> >>> >> >
>> >> >> >> >>>> >> >>> >>
>> >> >> >> >>>> >> >>> >
>> >> >> >> >>>> >> >>>
>> >> >> >> >>>> >> >>
>> >> >> >> >>>> >> >>
>> >> >> >> >>>> >> >
>> >> >> >> >>>> >>
>> >> >> >> >>>> >
>> >> >> >> >>>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>
>> >> >> >> >
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Nelson, try this fix:
https://repository.apache.org/content/repositories/snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-20161027.181519-7.tar.gz
Keep in mind that if you bounce a broker while a CAS is being processed in
a service, the client will not be able to get the input CAS back nor will
it get child CASes. Both the input and child CAS will be thrown away by the
service due to undeliverable destination (old temp queue). The code will
actually stop the Cas Multiplier from producing child CASes if it detects
an undeliverable destination. You need a process timeout on the client
side to deal with that. When a broker is bounced, both the client and a
service will detect a new broker instance, kill its old temp queue
listeners and create new ones so that subsequent processing can take place.
Jerry
On Thu, Oct 27, 2016 at 1:19 PM, nelson rivera <ne...@gmail.com>
wrote:
> Hi, I would like know if you have worked in the solution of this
> theme, related with recovered of queues in service cas multiplier,
> after restart uima-as broker, specifically temp queue. If you have a
> snapshot with this. Thanks
>
> 2016-09-28 10:34 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Nelson, yes I think there is a problem in the UIMA-AS. First, there is
> > evidence that a service
> > is trying to use a stale temp queue:
> >
> > tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> > Cannot use a Temporary destination from another Connection.
> >
> >
> > Also, I think there is another problem related to recovery of temp queues
> > in the service.
> > The service cas multiplier uses multiple queues:
> > 1) input queue - receives GetMeta and Process requests from a client
> > 2) temp queue - receives FreeCas requests from a client. This is only
> > created when a service
> > is a Cas Multiplier. This is used to free CASes in a cas multiplier.
> >
> > The input queue listener recovers the connection since it received an
> input
> > CAS. I think the
> > temp queue is not recovered and this leads to repeated retries.
> >
> > I will work on fixing this today and let you know when this is ready. I
> > will also create a new
> > snapshot for you to try.
> >
> > Thanks for finding these problems.
> >
> > - jerry
> >
> >
> > On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <
> nelsonrivera12@gmail.com>
> > wrote:
> >
> >> ok, only show you the result of last test.
> >> Now in my test with uima-as-2.9.0, after complete restart of the
> >> Broker UIMA-AS log show
> >>
> >> 09:51:01.731 - 19:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> 09:51:01.740 - 46:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> 09:51:01.747 - 15:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> >> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> >> 09:51:01.773 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 09:51:01.773 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 09:51:01.774 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException
> >> 09:51:01.774 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
> >> Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
> >> 09:51:01.777 - 19:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 09:51:01.777 - 19:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> WARNING: Jms Listener Failed. Endpoint:
> >> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
> >> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> >> Cannot use a Temporary destination from another Connection.
> >>
> >> this lines are continue to repeat in the log indefinitely. And after
> >> this have the same problem when send input Cas at the service
> >>
> >> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > Nelson, I will investigate your use case and let you know.
> >> >
> >> > -jerry
> >> >
> >> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera
> >> > <nelsonrivera12@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> Thank you for describe how UIMA-AS works, it is very helpful. But my
> >> >> scenario is not restart the broker while the uima-as service works on
> >> >> input CAS
> >> >>
> >> >> my scenario is
> >> >> 1- Start Broker and UIMA-AS aggregate service
> >> >>
> >> >> 2- Complete Restart of the Broker (without restart the service).
> >> >> UIMA-AS log show the lines that previously i said you:
> >> >> "Jms Listener Failed." ....
> >> >> "The Session is closed." ...
> >> >> "The Consumer is closed." ...
> >> >>
> >> >> I do not see in my uima-as log the lines that you say in your test
> >> >> after broker is bounced:
> >> >> Listeners reconnect:
> >> >> INFO | Successfully refreshed JMS Connection
> >> >> INFO | Successfully refreshed JMS Connection
> >> >> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
> >> >> the connection to the broker after restart.
> >> >>
> >> >> 4- Then in this point, after the broker was restarted completely,
> >> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
> >> >> petition UIMA-AS client receive the parent CAS, but not child CAS.
> >> >>
> >> >> In this moment i will trys with uima-as-2.9.0 snapshot
> >> >>
> >> >>
> >> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >> > Nelson, in my scenario I do have a Cas Multiplier which produces a
> >> >> > child
> >> >> > CAS for each input CAS. For the purpose of this
> >> >> > test its not important how many internal delegates there are in a
> >> >> pipeline.
> >> >> > The service is configured as a Cas Multiplier
> >> >> > and it does send child CASes when a broker is running.
> >> >> >
> >> >> > The problem is that once the input CAS is received, its reply queue
> >> >> > is
> >> >> > bound to a specific temp reply queue. When a broker is
> >> >> > bounced, that queue is not valid anymore and cant be used to return
> >> >> > child
> >> >> > CASes nor the input CAS. To guard against such
> >> >> > cases you need to use timeouts on the client side.
> >> >> >
> >> >> > Like I said in my previous email I dont get how an input CAS can be
> >> >> > returned back to the client after a broker bounce. This should
> >> >> > not happen since its temp queue does not exist in a new broker.
> >> >> >
> >> >> > I am still looking at this scenario as I don't like what I see in
> >> >> > the
> >> >> > service log. I think there might be a bug in the UIMA-AS but I am
> >> >> > still investigating. Bug or not, what I described above is how
> >> >> > UIMA-AS
> >> >> > works. You should not expect any replies from a service
> >> >> > working on a CAS that was sent through a broker that is bounced
> >> >> > before
> >> >> > a
> >> >> > reply.
> >> >> >
> >> >> > -jerry
> >> >> >
> >> >> >
> >> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
> >> >> > <nelsonrivera12@gmail.com
> >> >> >
> >> >> > wrote:
> >> >> >
> >> >> >> My service is a aggregate analysis engine that receives input CAS
> >> >> >> with the direcction of a directory of documents, the first
> >> >> >> component
> >> >> >> is a MultiplerCas that for each document generate a child CAS,
> each
> >> >> >> child CAS continues the flow that are other primitive annotators
> >> >> >> (primitive annotators in c++ that can throw exceptions), then
> >> >> >> exists
> >> >> >> other MultiplerCas that acts as Merger, waiting the complete
> >> >> >> processing of each child CAS and finally generate a child CAS
> where
> >> >> >> each child CAS previously processed is a view of CAS, this a child
> >> CAS
> >> >> >> that is composed of view continúa el flujo a otro AE, que espera
> un
> >> >> >> CAS de esta manera.
> >> >> >> I will send you my uima-as descriptor, to understand better my
> >> >> >> service. All delegates are co-located.
> >> >> >>
> >> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
> >> >> >> behaves
> >> >> >>
> >> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >> >> > I was able to stage this scenario in which an aggregate service
> >> >> >> > cas
> >> >> >> > multiplier receives a single input CAS and
> >> >> >> > produces a child CAS.
> >> >> >> >
> >> >> >> > Scenario:
> >> >> >> > 1) Service receives input CAS
> >> >> >> > 2) CasMultiplier generates a child CAS
> >> >> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> >> >> >> > 4) Broker is shutdown
> >> >> >> > 5) AE.process() awakes from sleep
> >> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
> >> >> >> > 7) Connection is invalid so the child CAS is dropped
> >> >> >> > 8) Broker is restarted
> >> >> >> > 9) Listeners reconnect:
> >> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> >> > 10) Broker log shows:
> >> >> >> > WARN Service - Async error occurred:
> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> >> > destination
> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> >> exist.
> >> >> >> > WARN Service - Async error occurred:
> >> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> >> > destination
> >> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> >> exist.
> >> >> >> >
> >> >> >> > The service failed to send a reply.Nothing is delivered to a
> >> >> >> > client
> >> >> >> > as
> >> >> >> > expected. This is the confusing part since you see a parent CAS
> >> >> >> > being
> >> >> >> sent
> >> >> >> > to a client. I just cant imagine how this is possible.
> >> >> >> >
> >> >> >> > When a service receives an input CAS in a message, it extracts a
> >> JMS
> >> >> >> > replyTo object which is a JMS temp queue and stores in an
> >> >> >> > internal
> >> >> >> > cache
> >> >> >> > until it is time to send a reply. Temp queues are transient and
> >> >> >> > unique
> >> >> >> each
> >> >> >> > time one is created.
> >> >> >> >
> >> >> >> > If the broker dies and subsequently restarts, the cached replyTo
> >> >> >> > queue
> >> >> >> > object is invalid (temp queue no longer exist) which leads to
> >> >> >> > "Async error occurred:
> >> >> >> > org.apache.activemq.DestinationDoesNotExistException" as shows
> >> >> >> > above.
> >> >> >> >
> >> >> >> > In the service log I see a bunch of these exceptions:
> >> >> >> >
> >> >> >> > 02:05:46.574 - 56:
> >> >> >> > org.apache.uima.adapter.jms.activemq.
> JmsEndpointConnection_impl.
> >> >> >> openChannel:
> >> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection
> To
> >> >> >> > Broker:
> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
> >> Silently
> >> >> >> > Retrying ...
> >> >> >> > 02:05:46.574 - 56:
> >> >> >> > org.apache.uima.adapter.jms.activemq.
> JmsEndpointConnection_impl.
> >> >> >> openChannel:
> >> >> >> > WARNING:
> >> >> >> > javax.jms.JMSException: Could not connect to broker URL:
> >> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0.
> Reason:
> >> >> >> > java.net.ConnectException: Connection refused
> >> >> >> >
> >> >> >> > This is *after* the service is restarted so there seems to be a
> >> >> problem
> >> >> >> > with reestablishing the connection to a new broker.
> >> >> >> > I am still investigating that part. I just am confused how you
> >> >> >> > are
> >> >> able
> >> >> >> to
> >> >> >> > see a reply in the client.
> >> >> >> >
> >> >> >> > -Jerry
> >> >> >> >
> >> >> >> >
> >> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
> >> uimaee@gmail.com>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> >> From what I understand there are two issues:
> >> >> >> >>
> >> >> >> >> 1) Why service JMS listeners are not reconnecting when a broker
> >> >> >> >> is
> >> >> >> >> bounced. This
> >> >> >> >> should be done by default. The 2.9.0 snapshot may address
> >> >> >> >> this
> >> >> >> >> since
> >> >> >> >> a
> >> >> >> >> listener
> >> >> >> >> uses newer AMQ jars (5.14.0)
> >> >> >> >>
> >> >> >> >> 2) What happens to a child CAS when a Broker disappears and the
> >> >> >> >> service
> >> >> >> >> wants
> >> >> >> >> to deliver it to a client that is not reachable. I need to
> >> >> >> >> investigate
> >> >> >> >> this part. I think
> >> >> >> >> such CAS is dropped but that should be reflected in a log
> >> >> >> >> and
> >> >> also
> >> >> >> >> the
> >> >> >> >> input CAS
> >> >> >> >> should be marked as failed. Again something for me to
> >> >> investigate.
> >> >> >> >>
> >> >> >> >> I will try to set up a test to see if I can create the
> >> >> >> >> scenario.It
> >> >> >> >> will
> >> >> >> >> take me a bit of time
> >> >> >> >> to set this up but I will let you know what happens.
> >> >> >> >>
> >> >> >> >> -jerry
> >> >> >> >>
> >> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
> >> >> >> >> <uimaee@gmail.com
> >> >> >
> >> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
> >> >> >> >>> https://repository.apache.org/content/repositories/snapshots
> >> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
> >> >> >> >>> 23.195233-6.tar.gz
> >> >> >> >>>
> >> >> >> >>> Can you briefly describe your service? Does your service have
> >> >> >> >>> any
> >> >> >> remote
> >> >> >> >>> delegates or all parts of the pipeline are in the same
> process?
> >> >> >> >>> Do you see and exceptions in the service log?
> >> >> >> >>>
> >> >> >> >>> -jerry
> >> >> >> >>>
> >> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
> >> >> >> >>> <ne...@gmail.com>
> >> >> >> >>> wrote:
> >> >> >> >>>
> >> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same
> error.
> >> >> >> >>>> not
> >> >> >> >>>> appears to be broker
> >> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
> >> >> >> >>>> stagnant
> >> >> >> >>>> at
> >> >> >> >>>> work
> >> >> >> >>>>
> >> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <
> uimaee@gmail.com
> >> >:
> >> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is related
> >> >> >> >>>> > to
> >> >> >> broker
> >> >> >> >>>> > issues.
> >> >> >> >>>> >
> >> >> >> >>>> > -jerry
> >> >> >> >>>> >
> >> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> >> >> >> >>>> nelsonrivera12@gmail.com>
> >> >> >> >>>> > wrote:
> >> >> >> >>>> >
> >> >> >> >>>> >> just like that, the parent CAS comes back but the child
> CAS
> >> >> >> >>>> >> doesnt.
> >> >> >> >>>> >>
> >> >> >> >>>> >> And the child CAS if processed, because the uima-as log
> >> >> >> >>>> >> shows
> >> >> >> >>>> >> the
> >> >> >> >>>> >> lines of the other annotators primitive of the flow
> >> processing
> >> >> >> >>>> >> the
> >> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
> >> >> >> >>>> >>
> >> >> >> >>>> >> uima-as log only shows
> >> >> >> >>>> >> "Jms Listener Failed. Endpoint:
> >> >> >> >>>> >> temp-queue://ID:localhsot-H81-
> M1-33610-1474481197190-1:1:1
> >> >> >> >>>> >> Managed
> >> >> >> >>>> By:
> >> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >> >> >>>> >> java.io.EOFException" ........
> >> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >> >>>> >> XClusterAnalyzerAggregate
> >> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
> >> >> >> >>>> >> closed..............
> >> >> >> >>>> >>
> >> >> >> >>>> >> The CAS is send to service after restart the Broker, and
> >> >> >> >>>> >> how
> >> >> >> >>>> >> say
> >> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i
> >> >> >> >>>> >> try
> >> >> >> quiesce
> >> >> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
> >> >> Service.
> >> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that
> is
> >> >> >> >>>> >> the
> >> >> >> >>>> >> child
> >> >> >> >>>> >> CAS.
> >> >> >> >>>> >>
> >> >> >> >>>> >> the client if can send more CASes to the service, but not
> >> >> receive
> >> >> >> >>>> >> response, for each CAS send after the one CAS that was
> send
> >> >> after
> >> >> >> >>>> >> restart broker (for which parent CAS comes back but the
> >> child
> >> >> >> >>>> >> CAS
> >> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
> >> >> >> >>>> >>
> >> >> >> >>>> >> More specific:
> >> >> >> >>>> >> scenario 1:
> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything
> >> >> >> >>>> >> is
> >> >> >> >>>> >> fine.
> >> >> >> >>>> >>
> >> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS
> >> >> >> >>>> >> log
> >> >> >> >>>> >> show
> >> >> >> >>>> >> :
> >> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> >> parent
> >> >> >> >>>> >> CAS
> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
> >> >> >> >>>> >> client.)
> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows :
> >> >> "Quiescing
> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >> >> >> >>>> >>
> >> >> >> >>>> >> scenario 2:
> >> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything
> >> >> >> >>>> >> is
> >> >> >> >>>> >> fine.
> >> >> >> >>>> >>
> >> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS
> >> >> >> >>>> >> log
> >> >> >> >>>> >> show
> >> >> >> >>>> >> :
> >> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> >> parent
> >> >> >> >>>> >> CAS
> >> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
> client)
> >> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
> >> service.(not
> >> >> >> comes
> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
> >> service.(not
> >> >> >> comes
> >> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >> >>>> >> If i try quiesce the Aggregate Service console shows:
> >> >> >> >>>> >> "Quiescing
> >> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >> >> >> >>>> >>
> >> >> >> >>>> >>
> >> >> >> >>>> >>
> >> >> >> >>>> >>
> >> >> >> >>>> >>
> >> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
> >> uimaee@gmail.com
> >> >> >:
> >> >> >> >>>> >> > When a parent CAS comes back (after broker is restored),
> >> can
> >> >> >> >>>> >> > the
> >> >> >> >>>> client
> >> >> >> >>>> >> > send more CASes to the service and receive responses? I
> >> >> >> >>>> >> > am
> >> >> >> >>>> >> > trying
> >> >> >> >>>> >> > to
> >> >> >> >>>> >> > determine what the real problem is here.
> >> >> >> >>>> >> >
> >> >> >> >>>> >> > 1) JMS Listener not recovering connection
> >> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when
> broker
> >> >> >> >>>> connection is
> >> >> >> >>>> >> > stale
> >> >> >> >>>> >> >
> >> >> >> >>>> >> > -jerry
> >> >> >> >>>> >> >
> >> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
> >> >> >> >>>> uimaee@gmail.com>
> >> >> >> >>>> >> > wrote:
> >> >> >> >>>> >> >
> >> >> >> >>>> >> >> Strange. You say that the parent CAS comes back but the
> >> >> >> >>>> >> >> child
> >> >> >> CAS
> >> >> >> >>>> >> doesnt.
> >> >> >> >>>> >> >> Are there any exceptions in the service log? Perhaps
> the
> >> >> >> >>>> >> >> service
> >> >> >> >>>> >> >> is
> >> >> >> >>>> >> >> unable
> >> >> >> >>>> >> >> to send the child CAS due to missing broker. There
> >> >> >> >>>> >> >> should
> >> >> >> >>>> >> >> be
> >> >> >> >>>> something
> >> >> >> >>>> >> in
> >> >> >> >>>> >> >> the uima-as log about this.
> >> >> >> >>>> >> >>
> >> >> >> >>>> >> >> -jerry
> >> >> >> >>>> >> >>
> >> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >> >> >>>> >> >> <ne...@gmail.com>
> >> >> >> >>>> >> >> wrote:
> >> >> >> >>>> >> >>
> >> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
> >> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS continue
> >> >> >> >>>> >> >>> in
> >> >> >> >>>> >> >>> the
> >> >> >> >>>> >> >>> flow,
> >> >> >> >>>> >> >>> that are other primitive annotators. And I expected
> >> >> >> >>>> >> >>> that
> >> >> >> >>>> >> >>> the
> >> >> >> >>>> >> >>> child
> >> >> >> >>>> >> >>> CAS
> >> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>> the scenario:
> >> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to
> >> >> >> >>>> >> >>> verify
> >> >> >> >>>> >> >>> everything is fine. The process is correct.
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>> 3- Restart Broker (without restart the service).
> >> >> >> >>>> >> >>> UIMA-AS
> >> >> log
> >> >> >> >>>> show :
> >> >> >> >>>> >> >>> "Jms Listener Failed." ....
> >> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child
> >> >> >> >>>> >> >>> CAS.
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>> Thank for you quick reply
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
> >> >> uimaee@gmail.com
> >> >> >> >:
> >> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
> >> >> Multiplier.
> >> >> >> >>>> Meaning
> >> >> >> >>>> >> >>> given a
> >> >> >> >>>> >> >>> > CAS it produces child CASes
> >> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what
> >> this
> >> >> >> >>>> service is
> >> >> >> >>>> >> >>> > supposed to do if the broker is running?
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> > Can you confirm the scenario:
> >> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >> >> >> >>>> >> >>> > 2) Service receives the CAS
> >> >> >> >>>> >> >>> > 3) Broker dies
> >> >> >> >>>> >> >>> > 4) Broker is restarted
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a
> >> >> >> >>>> >> >>> > reply
> >> >> >> >>>> >> >>> > when
> >> >> >> a
> >> >> >> >>>> >> >>> > broker
> >> >> >> >>>> >> >>> > is
> >> >> >> >>>> >> >>> > restarted?
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> > -Jerry
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
> >> >> >> >>>> >> >>> > wrote:
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>> >> I will try the variants that you say. I was
> >> >> >> >>>> >> >>> >> deepening
> >> >> >> >>>> >> >>> >> the
> >> >> >> >>>> problem:
> >> >> >> >>>> >> >>> >>
> >> >> >> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier,
> >> >> >> >>>> >> >>> >> the
> >> >> >> >>>> >> >>> >> original
> >> >> >> >>>> >> >>> >> CAS
> >> >> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier,
> >> >> >> >>>> >> >>> >> this
> >> >> >> >>>> generates
> >> >> >> >>>> >> >>> >> new
> >> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
> >> >> >> >>>> >> >>> >> Aggregate.
> >> >> >> >>>> >> >>> >>
> >> >> >> >>>> >> >>> >> After restart broker (without restart the service
> of
> >> >> >> >>>> >> >>> >> annotator)
> >> >> >> >>>> >> >>> >> and
> >> >> >> >>>> >> >>> >> send a CAS to process, the listener associated to
> >> >> >> >>>> >> >>> >> the
> >> >> >> client
> >> >> >> >>>> api
> >> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is
> >> >> >> >>>> >> >>> >> notified
> >> >> >> >>>> >> >>> >> when
> >> >> >> >>>> the
> >> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated
> in
> >> >> >> >>>> >> >>> >> the
> >> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >> >> >> >>>> >> >>> >>
> >> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but
> >> >> >> >>>> >> >>> >> how
> >> >> >> >>>> >> >>> >> the
> >> >> >> >>>> original
> >> >> >> >>>> >> >>> >> CAS
> >> >> >> >>>> >> >>> >> is notifies into the listener when finalizes, not
> >> have
> >> >> >> >>>> >> >>> >> problem
> >> >> >> >>>> >> >>> >> because is not expected for another new CAS.
> >> >> >> >>>> >> >>> >>
> >> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
> >> >> >> >>>> >> >>> >> <uimaee@gmail.com
> >> >> >> >>>> >:
> >> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest)
> >> >> >> >>>> >> >>> >> > and
> >> >> use
> >> >> >> >>>> >> >>> >> > that
> >> >> >> >>>> >> >>> >> > broker
> >> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with?
> >> >> >> >>>> >> >>> >> > There
> >> >> >> >>>> >> >>> >> > was
> >> >> >> >>>> >> >>> >> > a
> >> >> >> >>>> >> >>> >> > nasty
> >> >> >> >>>> >> >>> bug
> >> >> >> >>>> >> >>> >> > in
> >> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I
> >> >> >> >>>> >> >>> >> > am
> >> >> not
> >> >> >> >>>> sure if
> >> >> >> >>>> >> >>> >> > this
> >> >> >> >>>> >> >>> >> > is
> >> >> >> >>>> >> >>> >> > the cause but worth trying.
> >> >> >> >>>> >> >>> >> >
> >> >> >> >>>> >> >>> >> > You can download AMQ from here:
> >> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
> >> >> >> >>>> >> >>> >> >
> >> >> >> >>>> >> >>> >> > -Jerry
> >> >> >> >>>> >> >>> >> >
> >> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >> >> >> >>>> >> >>> >> > <ui...@gmail.com>
> >> >> >> >>>> >> >>> >> wrote:
> >> >> >> >>>> >> >>> >> >
> >> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage
> >> >> >> >>>> >> >>> >> >> this
> >> >> >> >>>> >> >>> >> >> with
> >> >> >> >>>> >> >>> >> >> UIMA-AS
> >> >> >> >>>> >> >>> >> >> 2.8.1.
> >> >> >> >>>> >> >>> >> I
> >> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code.
> >> >> >> >>>> >> >>> >> >> The
> >> >> most
> >> >> >> >>>> current
> >> >> >> >>>> >> >>> >> >> code
> >> >> >> >>>> >> >>> >> does
> >> >> >> >>>> >> >>> >> >> have newer broker.
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> After the service started, I stopped the broker,
> >> >> waited
> >> >> >> >>>> >> >>> >> >> a
> >> >> >> >>>> bit,
> >> >> >> >>>> >> and
> >> >> >> >>>> >> >>> >> >> restarted the broker.
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in
> >> >> >> >>>> >> >>> >> >> the
> >> >> log:
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
> >> >> activemq.
> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >> >> >>>> >> >>> >> >> Controller:
> >> >> >> >>>> >> >>> >> >> Meeting
> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
> >> >> >> >>>> >> >>> >> >> Endpoint:
> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
> >> >> activemq.
> >> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >> >> >>>> >> >>> >> >> Controller:
> >> >> >> >>>> >> >>> >> >> Meeting
> >> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
> >> >> >> >>>> >> >>> >> >> Endpoint:
> >> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
> >> tcp://localhost:61616
> >> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> ainer.onException:
> >> >> >> WARNING:
> >> >> >> >>>> >> Service:
> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> ainer.onException:
> >> >> >> WARNING:
> >> >> >> >>>> Jms
> >> >> >> >>>> >> >>> Listener
> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
> Managed
> >> >> >> >>>> >> >>> >> >> By:
> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> >> java.io.EOFException
> >> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> Unable
> >> >> To
> >> >> >> >>>> Connect
> >> >> >> >>>> >> To
> >> >> >> >>>> >> >>> >> >> Broker:
> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
> >> >> >> >>>> >> >>> >> >> ...
> >> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> ainer.onException:
> >> >> >> WARNING:
> >> >> >> >>>> >> Service:
> >> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> ainer.onException:
> >> >> >> WARNING:
> >> >> >> >>>> Jms
> >> >> >> >>>> >> >>> Listener
> >> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue
> Managed
> >> >> >> >>>> >> >>> >> >> By:
> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> >> java.io.EOFException
> >> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> Unable
> >> >> To
> >> >> >> >>>> Connect
> >> >> >> >>>> >> To
> >> >> >> >>>> >> >>> >> >> Broker:
> >> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
> >> >> >> >>>> >> >>> >> >> ...
> >> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> Recovered
> >> >> >> >>>> >> Connection
> >> >> >> >>>> >> >>> to
> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
> >> >> >> >>>> >> >>> >> >> Again
> >> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
> >> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> WARNING:
> >> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> >> Recovered
> >> >> >> >>>> >> Connection
> >> >> >> >>>> >> >>> to
> >> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
> >> >> >> >>>> >> >>> >> >> Again
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> -Jerry
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
> >> >> >> >>>> >> >>> >> >> wrote:
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
> >> happen?
> >> >> >> >>>> >> >>> >> >>>
> >> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
> >> >> >> >>>> uimaee@gmail.com
> >> >> >> >>>> >> >:
> >> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >>>> >> >>> >> >>> > -Jerry
> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson
> rivera
> >> >> >> >>>> >> >>> >> >>> > <
> >> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >> >> >>>> >> >>> >> >>> > wrote:
> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator
> >> >> >> >>>> >> >>> >> >>> >> as
> >> >> >> >>>> >> >>> >> >>> >> service,
> >> >> >> >>>> >> >>> >> >>> >> processing
> >> >> >> >>>> >> >>> >> >>> >> is
> >> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker is
> >> >> stopped
> >> >> >> >>>> >> >>> >> >>> >> and
> >> >> >> >>>> >> >>> >> >>> >> subsequently
> >> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below
> >> and
> >> >> >> after
> >> >> >> >>>> this
> >> >> >> >>>> >> >>> >> >>> >> the
> >> >> >> >>>> >> >>> >> >>> >> api
> >> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the
> >> >> >> >>>> >> >>> >> >>> >> listener
> >> >> when
> >> >> >> >>>> >> >>> >> >>> >> a
> >> >> >> >>>> cas
> >> >> >> >>>> >> >>> >> >>> >> is
> >> >> >> >>>> >> >>> >> >>> >> processed, any more.
> >> >> >> >>>> >> >>> >> >>> >> can help me?:
> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> Analyzer
> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >>>> >> >>> Exception
> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >>>> >> >>> javax.jms.JMSException:
> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
> >> >> >> >>>> >> >>> >> >>> >> Cluster
> >> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >> >>>> Retrying
> >> >> >> >>>> >> >>> >> >>> >> ...
> >> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
> >> >> >> >>>> >> >>> >> >>> >> Failed.
> >> >> >> >>>> Endpoint:
> >> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> >> >> >> >>>> M1-33610-1474481197190-1:1:1
> >> >> >> >>>> >> >>> Managed
> >> >> >> >>>> >> >>> >> >>> >> By:
> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
> >> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
> >> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
> >> >> >> >>>> >> >>> >> >>> >> Cluster
> >> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
> >> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> Analyzer
> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >>>> >> >>> Exception
> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.
> ConnectionClosedException:
> >> >> The
> >> >> >> >>>> >> connection
> >> >> >> >>>> >> >>> is
> >> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> >> Analyzer
> >> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >> >>>> >> >>> Exception
> >> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >> >>>> >> >>> nerCont
> >> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The
> Consumer
> >> is
> >> >> >> >>>> closed.
> >> >> >> >>>> >> >>> >> >>> >>
> >> >> >> >>>> >> >>> >> >>> >
> >> >> >> >>>> >> >>> >> >>>
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >>
> >> >> >> >>>> >> >>> >> >
> >> >> >> >>>> >> >>> >>
> >> >> >> >>>> >> >>> >
> >> >> >> >>>> >> >>>
> >> >> >> >>>> >> >>
> >> >> >> >>>> >> >>
> >> >> >> >>>> >> >
> >> >> >> >>>> >>
> >> >> >> >>>> >
> >> >> >> >>>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>
> >> >> >> >
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
Hi, I would like know if you have worked in the solution of this
theme, related with recovered of queues in service cas multiplier,
after restart uima-as broker, specifically temp queue. If you have a
snapshot with this. Thanks
2016-09-28 10:34 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Nelson, yes I think there is a problem in the UIMA-AS. First, there is
> evidence that a service
> is trying to use a stale temp queue:
>
> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> Cannot use a Temporary destination from another Connection.
>
>
> Also, I think there is another problem related to recovery of temp queues
> in the service.
> The service cas multiplier uses multiple queues:
> 1) input queue - receives GetMeta and Process requests from a client
> 2) temp queue - receives FreeCas requests from a client. This is only
> created when a service
> is a Cas Multiplier. This is used to free CASes in a cas multiplier.
>
> The input queue listener recovers the connection since it received an input
> CAS. I think the
> temp queue is not recovered and this leads to repeated retries.
>
> I will work on fixing this today and let you know when this is ready. I
> will also create a new
> snapshot for you to try.
>
> Thanks for finding these problems.
>
> - jerry
>
>
> On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> ok, only show you the result of last test.
>> Now in my test with uima-as-2.9.0, after complete restart of the
>> Broker UIMA-AS log show
>>
>> 09:51:01.731 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> 09:51:01.740 - 46:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> 09:51:01.747 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
>> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
>> 09:51:01.773 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> 09:51:01.773 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> 09:51:01.774 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 09:51:01.774 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
>> Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
>> 09:51:01.777 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
>> 09:51:01.777 - 19:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> WARNING: Jms Listener Failed. Endpoint:
>> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
>> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
>> Cannot use a Temporary destination from another Connection.
>>
>> this lines are continue to repeat in the log indefinitely. And after
>> this have the same problem when send input Cas at the service
>>
>> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Nelson, I will investigate your use case and let you know.
>> >
>> > -jerry
>> >
>> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera
>> > <nelsonrivera12@gmail.com
>> >
>> > wrote:
>> >
>> >> Thank you for describe how UIMA-AS works, it is very helpful. But my
>> >> scenario is not restart the broker while the uima-as service works on
>> >> input CAS
>> >>
>> >> my scenario is
>> >> 1- Start Broker and UIMA-AS aggregate service
>> >>
>> >> 2- Complete Restart of the Broker (without restart the service).
>> >> UIMA-AS log show the lines that previously i said you:
>> >> "Jms Listener Failed." ....
>> >> "The Session is closed." ...
>> >> "The Consumer is closed." ...
>> >>
>> >> I do not see in my uima-as log the lines that you say in your test
>> >> after broker is bounced:
>> >> Listeners reconnect:
>> >> INFO | Successfully refreshed JMS Connection
>> >> INFO | Successfully refreshed JMS Connection
>> >> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
>> >> the connection to the broker after restart.
>> >>
>> >> 4- Then in this point, after the broker was restarted completely,
>> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
>> >> petition UIMA-AS client receive the parent CAS, but not child CAS.
>> >>
>> >> In this moment i will trys with uima-as-2.9.0 snapshot
>> >>
>> >>
>> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > Nelson, in my scenario I do have a Cas Multiplier which produces a
>> >> > child
>> >> > CAS for each input CAS. For the purpose of this
>> >> > test its not important how many internal delegates there are in a
>> >> pipeline.
>> >> > The service is configured as a Cas Multiplier
>> >> > and it does send child CASes when a broker is running.
>> >> >
>> >> > The problem is that once the input CAS is received, its reply queue
>> >> > is
>> >> > bound to a specific temp reply queue. When a broker is
>> >> > bounced, that queue is not valid anymore and cant be used to return
>> >> > child
>> >> > CASes nor the input CAS. To guard against such
>> >> > cases you need to use timeouts on the client side.
>> >> >
>> >> > Like I said in my previous email I dont get how an input CAS can be
>> >> > returned back to the client after a broker bounce. This should
>> >> > not happen since its temp queue does not exist in a new broker.
>> >> >
>> >> > I am still looking at this scenario as I don't like what I see in
>> >> > the
>> >> > service log. I think there might be a bug in the UIMA-AS but I am
>> >> > still investigating. Bug or not, what I described above is how
>> >> > UIMA-AS
>> >> > works. You should not expect any replies from a service
>> >> > working on a CAS that was sent through a broker that is bounced
>> >> > before
>> >> > a
>> >> > reply.
>> >> >
>> >> > -jerry
>> >> >
>> >> >
>> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
>> >> > <nelsonrivera12@gmail.com
>> >> >
>> >> > wrote:
>> >> >
>> >> >> My service is a aggregate analysis engine that receives input CAS
>> >> >> with the direcction of a directory of documents, the first
>> >> >> component
>> >> >> is a MultiplerCas that for each document generate a child CAS, each
>> >> >> child CAS continues the flow that are other primitive annotators
>> >> >> (primitive annotators in c++ that can throw exceptions), then
>> >> >> exists
>> >> >> other MultiplerCas that acts as Merger, waiting the complete
>> >> >> processing of each child CAS and finally generate a child CAS where
>> >> >> each child CAS previously processed is a view of CAS, this a child
>> CAS
>> >> >> that is composed of view continúa el flujo a otro AE, que espera un
>> >> >> CAS de esta manera.
>> >> >> I will send you my uima-as descriptor, to understand better my
>> >> >> service. All delegates are co-located.
>> >> >>
>> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
>> >> >> behaves
>> >> >>
>> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >> > I was able to stage this scenario in which an aggregate service
>> >> >> > cas
>> >> >> > multiplier receives a single input CAS and
>> >> >> > produces a child CAS.
>> >> >> >
>> >> >> > Scenario:
>> >> >> > 1) Service receives input CAS
>> >> >> > 2) CasMultiplier generates a child CAS
>> >> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
>> >> >> > 4) Broker is shutdown
>> >> >> > 5) AE.process() awakes from sleep
>> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
>> >> >> > 7) Connection is invalid so the child CAS is dropped
>> >> >> > 8) Broker is restarted
>> >> >> > 9) Listeners reconnect:
>> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> > INFO | Successfully refreshed JMS Connection
>> >> >> > 10) Broker log shows:
>> >> >> > WARN Service - Async error occurred:
>> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> > destination
>> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
>> exist.
>> >> >> > WARN Service - Async error occurred:
>> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> >> > destination
>> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
>> exist.
>> >> >> >
>> >> >> > The service failed to send a reply.Nothing is delivered to a
>> >> >> > client
>> >> >> > as
>> >> >> > expected. This is the confusing part since you see a parent CAS
>> >> >> > being
>> >> >> sent
>> >> >> > to a client. I just cant imagine how this is possible.
>> >> >> >
>> >> >> > When a service receives an input CAS in a message, it extracts a
>> JMS
>> >> >> > replyTo object which is a JMS temp queue and stores in an
>> >> >> > internal
>> >> >> > cache
>> >> >> > until it is time to send a reply. Temp queues are transient and
>> >> >> > unique
>> >> >> each
>> >> >> > time one is created.
>> >> >> >
>> >> >> > If the broker dies and subsequently restarts, the cached replyTo
>> >> >> > queue
>> >> >> > object is invalid (temp queue no longer exist) which leads to
>> >> >> > "Async error occurred:
>> >> >> > org.apache.activemq.DestinationDoesNotExistException" as shows
>> >> >> > above.
>> >> >> >
>> >> >> > In the service log I see a bunch of these exceptions:
>> >> >> >
>> >> >> > 02:05:46.574 - 56:
>> >> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> >> >> openChannel:
>> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To
>> >> >> > Broker:
>> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
>> Silently
>> >> >> > Retrying ...
>> >> >> > 02:05:46.574 - 56:
>> >> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> >> >> openChannel:
>> >> >> > WARNING:
>> >> >> > javax.jms.JMSException: Could not connect to broker URL:
>> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
>> >> >> > java.net.ConnectException: Connection refused
>> >> >> >
>> >> >> > This is *after* the service is restarted so there seems to be a
>> >> problem
>> >> >> > with reestablishing the connection to a new broker.
>> >> >> > I am still investigating that part. I just am confused how you
>> >> >> > are
>> >> able
>> >> >> to
>> >> >> > see a reply in the client.
>> >> >> >
>> >> >> > -Jerry
>> >> >> >
>> >> >> >
>> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
>> uimaee@gmail.com>
>> >> >> wrote:
>> >> >> >
>> >> >> >> From what I understand there are two issues:
>> >> >> >>
>> >> >> >> 1) Why service JMS listeners are not reconnecting when a broker
>> >> >> >> is
>> >> >> >> bounced. This
>> >> >> >> should be done by default. The 2.9.0 snapshot may address
>> >> >> >> this
>> >> >> >> since
>> >> >> >> a
>> >> >> >> listener
>> >> >> >> uses newer AMQ jars (5.14.0)
>> >> >> >>
>> >> >> >> 2) What happens to a child CAS when a Broker disappears and the
>> >> >> >> service
>> >> >> >> wants
>> >> >> >> to deliver it to a client that is not reachable. I need to
>> >> >> >> investigate
>> >> >> >> this part. I think
>> >> >> >> such CAS is dropped but that should be reflected in a log
>> >> >> >> and
>> >> also
>> >> >> >> the
>> >> >> >> input CAS
>> >> >> >> should be marked as failed. Again something for me to
>> >> investigate.
>> >> >> >>
>> >> >> >> I will try to set up a test to see if I can create the
>> >> >> >> scenario.It
>> >> >> >> will
>> >> >> >> take me a bit of time
>> >> >> >> to set this up but I will let you know what happens.
>> >> >> >>
>> >> >> >> -jerry
>> >> >> >>
>> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
>> >> >> >> <uimaee@gmail.com
>> >> >
>> >> >> >> wrote:
>> >> >> >>
>> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> >> >> >>> https://repository.apache.org/content/repositories/snapshots
>> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> >> >> >>> 23.195233-6.tar.gz
>> >> >> >>>
>> >> >> >>> Can you briefly describe your service? Does your service have
>> >> >> >>> any
>> >> >> remote
>> >> >> >>> delegates or all parts of the pipeline are in the same process?
>> >> >> >>> Do you see and exceptions in the service log?
>> >> >> >>>
>> >> >> >>> -jerry
>> >> >> >>>
>> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>> >> >> >>> <ne...@gmail.com>
>> >> >> >>> wrote:
>> >> >> >>>
>> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same error.
>> >> >> >>>> not
>> >> >> >>>> appears to be broker
>> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
>> >> >> >>>> stagnant
>> >> >> >>>> at
>> >> >> >>>> work
>> >> >> >>>>
>> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>> >:
>> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is related
>> >> >> >>>> > to
>> >> >> broker
>> >> >> >>>> > issues.
>> >> >> >>>> >
>> >> >> >>>> > -jerry
>> >> >> >>>> >
>> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> >> >> >>>> nelsonrivera12@gmail.com>
>> >> >> >>>> > wrote:
>> >> >> >>>> >
>> >> >> >>>> >> just like that, the parent CAS comes back but the child CAS
>> >> >> >>>> >> doesnt.
>> >> >> >>>> >>
>> >> >> >>>> >> And the child CAS if processed, because the uima-as log
>> >> >> >>>> >> shows
>> >> >> >>>> >> the
>> >> >> >>>> >> lines of the other annotators primitive of the flow
>> processing
>> >> >> >>>> >> the
>> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
>> >> >> >>>> >>
>> >> >> >>>> >> uima-as log only shows
>> >> >> >>>> >> "Jms Listener Failed. Endpoint:
>> >> >> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1
>> >> >> >>>> >> Managed
>> >> >> >>>> By:
>> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >> >>>> >> java.io.EOFException" ........
>> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >>>> >> XClusterAnalyzerAggregate
>> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
>> >> >> >>>> >> closed..............
>> >> >> >>>> >>
>> >> >> >>>> >> The CAS is send to service after restart the Broker, and
>> >> >> >>>> >> how
>> >> >> >>>> >> say
>> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i
>> >> >> >>>> >> try
>> >> >> quiesce
>> >> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
>> >> Service.
>> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that is
>> >> >> >>>> >> the
>> >> >> >>>> >> child
>> >> >> >>>> >> CAS.
>> >> >> >>>> >>
>> >> >> >>>> >> the client if can send more CASes to the service, but not
>> >> receive
>> >> >> >>>> >> response, for each CAS send after the one CAS that was send
>> >> after
>> >> >> >>>> >> restart broker (for which parent CAS comes back but the
>> child
>> >> >> >>>> >> CAS
>> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
>> >> >> >>>> >>
>> >> >> >>>> >> More specific:
>> >> >> >>>> >> scenario 1:
>> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything
>> >> >> >>>> >> is
>> >> >> >>>> >> fine.
>> >> >> >>>> >>
>> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS
>> >> >> >>>> >> log
>> >> >> >>>> >> show
>> >> >> >>>> >> :
>> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
>> parent
>> >> >> >>>> >> CAS
>> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as
>> >> >> >>>> >> client.)
>> >> >> >>>> >> If i try quiesce the Aggregate Service console shows :
>> >> "Quiescing
>> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >> >> >>>> >>
>> >> >> >>>> >> scenario 2:
>> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything
>> >> >> >>>> >> is
>> >> >> >>>> >> fine.
>> >> >> >>>> >>
>> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS
>> >> >> >>>> >> log
>> >> >> >>>> >> show
>> >> >> >>>> >> :
>> >> >> >>>> >> "Jms Listener Failed." ....
>> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
>> parent
>> >> >> >>>> >> CAS
>> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
>> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
>> service.(not
>> >> >> comes
>> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
>> service.(not
>> >> >> comes
>> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >> >>>> >> If i try quiesce the Aggregate Service console shows:
>> >> >> >>>> >> "Quiescing
>> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
>> uimaee@gmail.com
>> >> >:
>> >> >> >>>> >> > When a parent CAS comes back (after broker is restored),
>> can
>> >> >> >>>> >> > the
>> >> >> >>>> client
>> >> >> >>>> >> > send more CASes to the service and receive responses? I
>> >> >> >>>> >> > am
>> >> >> >>>> >> > trying
>> >> >> >>>> >> > to
>> >> >> >>>> >> > determine what the real problem is here.
>> >> >> >>>> >> >
>> >> >> >>>> >> > 1) JMS Listener not recovering connection
>> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
>> >> >> >>>> connection is
>> >> >> >>>> >> > stale
>> >> >> >>>> >> >
>> >> >> >>>> >> > -jerry
>> >> >> >>>> >> >
>> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>> >> >> >>>> uimaee@gmail.com>
>> >> >> >>>> >> > wrote:
>> >> >> >>>> >> >
>> >> >> >>>> >> >> Strange. You say that the parent CAS comes back but the
>> >> >> >>>> >> >> child
>> >> >> CAS
>> >> >> >>>> >> doesnt.
>> >> >> >>>> >> >> Are there any exceptions in the service log? Perhaps the
>> >> >> >>>> >> >> service
>> >> >> >>>> >> >> is
>> >> >> >>>> >> >> unable
>> >> >> >>>> >> >> to send the child CAS due to missing broker. There
>> >> >> >>>> >> >> should
>> >> >> >>>> >> >> be
>> >> >> >>>> something
>> >> >> >>>> >> in
>> >> >> >>>> >> >> the uima-as log about this.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> -jerry
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> >> >>>> >> >> <ne...@gmail.com>
>> >> >> >>>> >> >> wrote:
>> >> >> >>>> >> >>
>> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
>> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS continue
>> >> >> >>>> >> >>> in
>> >> >> >>>> >> >>> the
>> >> >> >>>> >> >>> flow,
>> >> >> >>>> >> >>> that are other primitive annotators. And I expected
>> >> >> >>>> >> >>> that
>> >> >> >>>> >> >>> the
>> >> >> >>>> >> >>> child
>> >> >> >>>> >> >>> CAS
>> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> the scenario:
>> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to
>> >> >> >>>> >> >>> verify
>> >> >> >>>> >> >>> everything is fine. The process is correct.
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> 3- Restart Broker (without restart the service).
>> >> >> >>>> >> >>> UIMA-AS
>> >> log
>> >> >> >>>> show :
>> >> >> >>>> >> >>> "Jms Listener Failed." ....
>> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child
>> >> >> >>>> >> >>> CAS.
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> Thank for you quick reply
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
>> >> uimaee@gmail.com
>> >> >> >:
>> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
>> >> Multiplier.
>> >> >> >>>> Meaning
>> >> >> >>>> >> >>> given a
>> >> >> >>>> >> >>> > CAS it produces child CASes
>> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what
>> this
>> >> >> >>>> service is
>> >> >> >>>> >> >>> > supposed to do if the broker is running?
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> > Can you confirm the scenario:
>> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >> >> >>>> >> >>> > 2) Service receives the CAS
>> >> >> >>>> >> >>> > 3) Broker dies
>> >> >> >>>> >> >>> > 4) Broker is restarted
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a
>> >> >> >>>> >> >>> > reply
>> >> >> >>>> >> >>> > when
>> >> >> a
>> >> >> >>>> >> >>> > broker
>> >> >> >>>> >> >>> > is
>> >> >> >>>> >> >>> > restarted?
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> > -Jerry
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >>>> >> >>> > wrote:
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>> >> I will try the variants that you say. I was
>> >> >> >>>> >> >>> >> deepening
>> >> >> >>>> >> >>> >> the
>> >> >> >>>> problem:
>> >> >> >>>> >> >>> >>
>> >> >> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier,
>> >> >> >>>> >> >>> >> the
>> >> >> >>>> >> >>> >> original
>> >> >> >>>> >> >>> >> CAS
>> >> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier,
>> >> >> >>>> >> >>> >> this
>> >> >> >>>> generates
>> >> >> >>>> >> >>> >> new
>> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
>> >> >> >>>> >> >>> >> Aggregate.
>> >> >> >>>> >> >>> >>
>> >> >> >>>> >> >>> >> After restart broker (without restart the service of
>> >> >> >>>> >> >>> >> annotator)
>> >> >> >>>> >> >>> >> and
>> >> >> >>>> >> >>> >> send a CAS to process, the listener associated to
>> >> >> >>>> >> >>> >> the
>> >> >> client
>> >> >> >>>> api
>> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is
>> >> >> >>>> >> >>> >> notified
>> >> >> >>>> >> >>> >> when
>> >> >> >>>> the
>> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in
>> >> >> >>>> >> >>> >> the
>> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >> >> >>>> >> >>> >>
>> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but
>> >> >> >>>> >> >>> >> how
>> >> >> >>>> >> >>> >> the
>> >> >> >>>> original
>> >> >> >>>> >> >>> >> CAS
>> >> >> >>>> >> >>> >> is notifies into the listener when finalizes, not
>> have
>> >> >> >>>> >> >>> >> problem
>> >> >> >>>> >> >>> >> because is not expected for another new CAS.
>> >> >> >>>> >> >>> >>
>> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>> >> >> >>>> >> >>> >> <uimaee@gmail.com
>> >> >> >>>> >:
>> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest)
>> >> >> >>>> >> >>> >> > and
>> >> use
>> >> >> >>>> >> >>> >> > that
>> >> >> >>>> >> >>> >> > broker
>> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with?
>> >> >> >>>> >> >>> >> > There
>> >> >> >>>> >> >>> >> > was
>> >> >> >>>> >> >>> >> > a
>> >> >> >>>> >> >>> >> > nasty
>> >> >> >>>> >> >>> bug
>> >> >> >>>> >> >>> >> > in
>> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I
>> >> >> >>>> >> >>> >> > am
>> >> not
>> >> >> >>>> sure if
>> >> >> >>>> >> >>> >> > this
>> >> >> >>>> >> >>> >> > is
>> >> >> >>>> >> >>> >> > the cause but worth trying.
>> >> >> >>>> >> >>> >> >
>> >> >> >>>> >> >>> >> > You can download AMQ from here:
>> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
>> >> >> >>>> >> >>> >> >
>> >> >> >>>> >> >>> >> > -Jerry
>> >> >> >>>> >> >>> >> >
>> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>> >> >> >>>> >> >>> >> > <ui...@gmail.com>
>> >> >> >>>> >> >>> >> wrote:
>> >> >> >>>> >> >>> >> >
>> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage
>> >> >> >>>> >> >>> >> >> this
>> >> >> >>>> >> >>> >> >> with
>> >> >> >>>> >> >>> >> >> UIMA-AS
>> >> >> >>>> >> >>> >> >> 2.8.1.
>> >> >> >>>> >> >>> >> I
>> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code.
>> >> >> >>>> >> >>> >> >> The
>> >> most
>> >> >> >>>> current
>> >> >> >>>> >> >>> >> >> code
>> >> >> >>>> >> >>> >> does
>> >> >> >>>> >> >>> >> >> have newer broker.
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> After the service started, I stopped the broker,
>> >> waited
>> >> >> >>>> >> >>> >> >> a
>> >> >> >>>> bit,
>> >> >> >>>> >> and
>> >> >> >>>> >> >>> >> >> restarted the broker.
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in
>> >> >> >>>> >> >>> >> >> the
>> >> log:
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
>> >> activemq.
>> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
>> >> activemq.
>> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >> >> >>>> >> >>> >> >> Controller:
>> >> >> >>>> >> >>> >> >> Meeting
>> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on
>> >> >> >>>> >> >>> >> >> Endpoint:
>> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
>> tcp://localhost:61616
>> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> >> WARNING:
>> >> >> >>>> >> Service:
>> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> >> WARNING:
>> >> >> >>>> Jms
>> >> >> >>>> >> >>> Listener
>> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
>> >> >> >>>> >> >>> >> >> By:
>> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> java.io.EOFException
>> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> WARNING:
>> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> Unable
>> >> To
>> >> >> >>>> Connect
>> >> >> >>>> >> To
>> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
>> >> >> >>>> >> >>> >> >> ...
>> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> >> WARNING:
>> >> >> >>>> >> Service:
>> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> >> WARNING:
>> >> >> >>>> Jms
>> >> >> >>>> >> >>> Listener
>> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
>> >> >> >>>> >> >>> >> >> By:
>> >> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
>> java.io.EOFException
>> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> WARNING:
>> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> Unable
>> >> To
>> >> >> >>>> Connect
>> >> >> >>>> >> To
>> >> >> >>>> >> >>> >> >> Broker:
>> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
>> >> >> >>>> >> >>> >> >> ...
>> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> WARNING:
>> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> Recovered
>> >> >> >>>> >> Connection
>> >> >> >>>> >> >>> to
>> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
>> >> >> >>>> >> >>> >> >> Again
>> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
>> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >> >>>> ainer.handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> WARNING:
>> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> >> Recovered
>> >> >> >>>> >> Connection
>> >> >> >>>> >> >>> to
>> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
>> >> >> >>>> >> >>> >> >> Again
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> -Jerry
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
>> >> >> >>>> >> >>> >> >> wrote:
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
>> happen?
>> >> >> >>>> >> >>> >> >>>
>> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>> >> >> >>>> uimaee@gmail.com
>> >> >> >>>> >> >:
>> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >> >> >>>> >> >>> >> >>> >
>> >> >> >>>> >> >>> >> >>> > -Jerry
>> >> >> >>>> >> >>> >> >>> >
>> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera
>> >> >> >>>> >> >>> >> >>> > <
>> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >> >> >>>> >> >>> >> >>> > wrote:
>> >> >> >>>> >> >>> >> >>> >
>> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator
>> >> >> >>>> >> >>> >> >>> >> as
>> >> >> >>>> >> >>> >> >>> >> service,
>> >> >> >>>> >> >>> >> >>> >> processing
>> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker is
>> >> stopped
>> >> >> >>>> >> >>> >> >>> >> and
>> >> >> >>>> >> >>> >> >>> >> subsequently
>> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below
>> and
>> >> >> after
>> >> >> >>>> this
>> >> >> >>>> >> >>> >> >>> >> the
>> >> >> >>>> >> >>> >> >>> >> api
>> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the
>> >> >> >>>> >> >>> >> >>> >> listener
>> >> when
>> >> >> >>>> >> >>> >> >>> >> a
>> >> >> >>>> cas
>> >> >> >>>> >> >>> >> >>> >> is
>> >> >> >>>> >> >>> >> >>> >> processed, any more.
>> >> >> >>>> >> >>> >> >>> >> can help me?:
>> >> >> >>>> >> >>> >> >>> >>
>> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> Analyzer
>> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >>>> >> >>> Exception
>> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >>>> >> >>> javax.jms.JMSException:
>> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
>> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >>>> Retrying
>> >> >> >>>> >> >>> >> >>> >> ...
>> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
>> >> >> >>>> >> >>> >> >>> >> Failed.
>> >> >> >>>> Endpoint:
>> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> >> >> >>>> M1-33610-1474481197190-1:1:1
>> >> >> >>>> >> >>> Managed
>> >> >> >>>> >> >>> >> >>> >> By:
>> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
>> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
>> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> >> >>> nerContainer.
>> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
>> >> >> >>>> >> >>> >> >>> >> Cluster
>> >> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >> >>>> >> >>> >> >>> >> Listener
>> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
>> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >> >>>> >> >>> >> >>> >>
>> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> Analyzer
>> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >>>> >> >>> Exception
>> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException:
>> >> The
>> >> >> >>>> >> connection
>> >> >> >>>> >> >>> is
>> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
>> Analyzer
>> >> >> >>>> >> >>> >> >>> >> Runtime
>> >> >> >>>> >> >>> Exception
>> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >> >>>> >> >>> nerCont
>> >> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer
>> is
>> >> >> >>>> closed.
>> >> >> >>>> >> >>> >> >>> >>
>> >> >> >>>> >> >>> >> >>> >
>> >> >> >>>> >> >>> >> >>>
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >>
>> >> >> >>>> >> >>> >> >
>> >> >> >>>> >> >>> >>
>> >> >> >>>> >> >>> >
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>
>> >> >> >>>> >> >>
>> >> >> >>>> >> >
>> >> >> >>>> >>
>> >> >> >>>> >
>> >> >> >>>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Nelson, yes I think there is a problem in the UIMA-AS. First, there is
evidence that a service
is trying to use a stale temp queue:
tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
Cannot use a Temporary destination from another Connection.
Also, I think there is another problem related to recovery of temp queues
in the service.
The service cas multiplier uses multiple queues:
1) input queue - receives GetMeta and Process requests from a client
2) temp queue - receives FreeCas requests from a client. This is only
created when a service
is a Cas Multiplier. This is used to free CASes in a cas multiplier.
The input queue listener recovers the connection since it received an input
CAS. I think the
temp queue is not recovered and this leads to repeated retries.
I will work on fixing this today and let you know when this is ready. I
will also create a new
snapshot for you to try.
Thanks for finding these problems.
- jerry
On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <ne...@gmail.com>
wrote:
> ok, only show you the result of last test.
> Now in my test with uima-as-2.9.0, after complete restart of the
> Broker UIMA-AS log show
>
> 09:51:01.731 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.740 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.747 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.773 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.773 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.774 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 09:51:01.774 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
> 09:51:01.777 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.777 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint:
> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> Cannot use a Temporary destination from another Connection.
>
> this lines are continue to repeat in the log indefinitely. And after
> this have the same problem when send input Cas at the service
>
> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Nelson, I will investigate your use case and let you know.
> >
> > -jerry
> >
> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera <nelsonrivera12@gmail.com
> >
> > wrote:
> >
> >> Thank you for describe how UIMA-AS works, it is very helpful. But my
> >> scenario is not restart the broker while the uima-as service works on
> >> input CAS
> >>
> >> my scenario is
> >> 1- Start Broker and UIMA-AS aggregate service
> >>
> >> 2- Complete Restart of the Broker (without restart the service).
> >> UIMA-AS log show the lines that previously i said you:
> >> "Jms Listener Failed." ....
> >> "The Session is closed." ...
> >> "The Consumer is closed." ...
> >>
> >> I do not see in my uima-as log the lines that you say in your test
> >> after broker is bounced:
> >> Listeners reconnect:
> >> INFO | Successfully refreshed JMS Connection
> >> INFO | Successfully refreshed JMS Connection
> >> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
> >> the connection to the broker after restart.
> >>
> >> 4- Then in this point, after the broker was restarted completely,
> >> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
> >> petition UIMA-AS client receive the parent CAS, but not child CAS.
> >>
> >> In this moment i will trys with uima-as-2.9.0 snapshot
> >>
> >>
> >> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > Nelson, in my scenario I do have a Cas Multiplier which produces a
> >> > child
> >> > CAS for each input CAS. For the purpose of this
> >> > test its not important how many internal delegates there are in a
> >> pipeline.
> >> > The service is configured as a Cas Multiplier
> >> > and it does send child CASes when a broker is running.
> >> >
> >> > The problem is that once the input CAS is received, its reply queue is
> >> > bound to a specific temp reply queue. When a broker is
> >> > bounced, that queue is not valid anymore and cant be used to return
> >> > child
> >> > CASes nor the input CAS. To guard against such
> >> > cases you need to use timeouts on the client side.
> >> >
> >> > Like I said in my previous email I dont get how an input CAS can be
> >> > returned back to the client after a broker bounce. This should
> >> > not happen since its temp queue does not exist in a new broker.
> >> >
> >> > I am still looking at this scenario as I don't like what I see in the
> >> > service log. I think there might be a bug in the UIMA-AS but I am
> >> > still investigating. Bug or not, what I described above is how UIMA-AS
> >> > works. You should not expect any replies from a service
> >> > working on a CAS that was sent through a broker that is bounced before
> >> > a
> >> > reply.
> >> >
> >> > -jerry
> >> >
> >> >
> >> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
> >> > <nelsonrivera12@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> My service is a aggregate analysis engine that receives input CAS
> >> >> with the direcction of a directory of documents, the first component
> >> >> is a MultiplerCas that for each document generate a child CAS, each
> >> >> child CAS continues the flow that are other primitive annotators
> >> >> (primitive annotators in c++ that can throw exceptions), then exists
> >> >> other MultiplerCas that acts as Merger, waiting the complete
> >> >> processing of each child CAS and finally generate a child CAS where
> >> >> each child CAS previously processed is a view of CAS, this a child
> CAS
> >> >> that is composed of view continúa el flujo a otro AE, que espera un
> >> >> CAS de esta manera.
> >> >> I will send you my uima-as descriptor, to understand better my
> >> >> service. All delegates are co-located.
> >> >>
> >> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
> >> >> behaves
> >> >>
> >> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >> > I was able to stage this scenario in which an aggregate service cas
> >> >> > multiplier receives a single input CAS and
> >> >> > produces a child CAS.
> >> >> >
> >> >> > Scenario:
> >> >> > 1) Service receives input CAS
> >> >> > 2) CasMultiplier generates a child CAS
> >> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> >> >> > 4) Broker is shutdown
> >> >> > 5) AE.process() awakes from sleep
> >> >> > 6) Uima-AS tries to deliver child CAS to a remote client
> >> >> > 7) Connection is invalid so the child CAS is dropped
> >> >> > 8) Broker is restarted
> >> >> > 9) Listeners reconnect:
> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> > INFO | Successfully refreshed JMS Connection
> >> >> > 10) Broker log shows:
> >> >> > WARN Service - Async error occurred:
> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> > destination
> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> exist.
> >> >> > WARN Service - Async error occurred:
> >> >> > org.apache.activemq.DestinationDoesNotExistException: The
> >> >> > destination
> >> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not
> exist.
> >> >> >
> >> >> > The service failed to send a reply.Nothing is delivered to a client
> >> >> > as
> >> >> > expected. This is the confusing part since you see a parent CAS
> >> >> > being
> >> >> sent
> >> >> > to a client. I just cant imagine how this is possible.
> >> >> >
> >> >> > When a service receives an input CAS in a message, it extracts a
> JMS
> >> >> > replyTo object which is a JMS temp queue and stores in an internal
> >> >> > cache
> >> >> > until it is time to send a reply. Temp queues are transient and
> >> >> > unique
> >> >> each
> >> >> > time one is created.
> >> >> >
> >> >> > If the broker dies and subsequently restarts, the cached replyTo
> >> >> > queue
> >> >> > object is invalid (temp queue no longer exist) which leads to
> >> >> > "Async error occurred:
> >> >> > org.apache.activemq.DestinationDoesNotExistException" as shows
> >> >> > above.
> >> >> >
> >> >> > In the service log I see a bunch of these exceptions:
> >> >> >
> >> >> > 02:05:46.574 - 56:
> >> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> >> >> openChannel:
> >> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To
> >> >> > Broker:
> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 -
> Silently
> >> >> > Retrying ...
> >> >> > 02:05:46.574 - 56:
> >> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> >> >> openChannel:
> >> >> > WARNING:
> >> >> > javax.jms.JMSException: Could not connect to broker URL:
> >> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
> >> >> > java.net.ConnectException: Connection refused
> >> >> >
> >> >> > This is *after* the service is restarted so there seems to be a
> >> problem
> >> >> > with reestablishing the connection to a new broker.
> >> >> > I am still investigating that part. I just am confused how you are
> >> able
> >> >> to
> >> >> > see a reply in the client.
> >> >> >
> >> >> > -Jerry
> >> >> >
> >> >> >
> >> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <
> uimaee@gmail.com>
> >> >> wrote:
> >> >> >
> >> >> >> From what I understand there are two issues:
> >> >> >>
> >> >> >> 1) Why service JMS listeners are not reconnecting when a broker is
> >> >> >> bounced. This
> >> >> >> should be done by default. The 2.9.0 snapshot may address this
> >> >> >> since
> >> >> >> a
> >> >> >> listener
> >> >> >> uses newer AMQ jars (5.14.0)
> >> >> >>
> >> >> >> 2) What happens to a child CAS when a Broker disappears and the
> >> >> >> service
> >> >> >> wants
> >> >> >> to deliver it to a client that is not reachable. I need to
> >> >> >> investigate
> >> >> >> this part. I think
> >> >> >> such CAS is dropped but that should be reflected in a log and
> >> also
> >> >> >> the
> >> >> >> input CAS
> >> >> >> should be marked as failed. Again something for me to
> >> investigate.
> >> >> >>
> >> >> >> I will try to set up a test to see if I can create the scenario.It
> >> >> >> will
> >> >> >> take me a bit of time
> >> >> >> to set this up but I will let you know what happens.
> >> >> >>
> >> >> >> -jerry
> >> >> >>
> >> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
> >> >> >> <uimaee@gmail.com
> >> >
> >> >> >> wrote:
> >> >> >>
> >> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
> >> >> >>> https://repository.apache.org/content/repositories/snapshots
> >> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
> >> >> >>> 23.195233-6.tar.gz
> >> >> >>>
> >> >> >>> Can you briefly describe your service? Does your service have any
> >> >> remote
> >> >> >>> delegates or all parts of the pipeline are in the same process?
> >> >> >>> Do you see and exceptions in the service log?
> >> >> >>>
> >> >> >>> -jerry
> >> >> >>>
> >> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
> >> >> >>> <ne...@gmail.com>
> >> >> >>> wrote:
> >> >> >>>
> >> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same error.
> >> >> >>>> not
> >> >> >>>> appears to be broker
> >> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
> >> >> >>>> stagnant
> >> >> >>>> at
> >> >> >>>> work
> >> >> >>>>
> >> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
> >:
> >> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is related to
> >> >> broker
> >> >> >>>> > issues.
> >> >> >>>> >
> >> >> >>>> > -jerry
> >> >> >>>> >
> >> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> >> >> >>>> nelsonrivera12@gmail.com>
> >> >> >>>> > wrote:
> >> >> >>>> >
> >> >> >>>> >> just like that, the parent CAS comes back but the child CAS
> >> >> >>>> >> doesnt.
> >> >> >>>> >>
> >> >> >>>> >> And the child CAS if processed, because the uima-as log
> >> >> >>>> >> shows
> >> >> >>>> >> the
> >> >> >>>> >> lines of the other annotators primitive of the flow
> processing
> >> >> >>>> >> the
> >> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
> >> >> >>>> >>
> >> >> >>>> >> uima-as log only shows
> >> >> >>>> >> "Jms Listener Failed. Endpoint:
> >> >> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1
> >> >> >>>> >> Managed
> >> >> >>>> By:
> >> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >> >>>> >> java.io.EOFException" ........
> >> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >>>> >> XClusterAnalyzerAggregate
> >> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
> >> >> >>>> >> closed..............
> >> >> >>>> >>
> >> >> >>>> >> The CAS is send to service after restart the Broker, and how
> >> >> >>>> >> say
> >> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i try
> >> >> quiesce
> >> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
> >> Service.
> >> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that is
> >> >> >>>> >> the
> >> >> >>>> >> child
> >> >> >>>> >> CAS.
> >> >> >>>> >>
> >> >> >>>> >> the client if can send more CASes to the service, but not
> >> receive
> >> >> >>>> >> response, for each CAS send after the one CAS that was send
> >> after
> >> >> >>>> >> restart broker (for which parent CAS comes back but the
> child
> >> >> >>>> >> CAS
> >> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
> >> >> >>>> >>
> >> >> >>>> >> More specific:
> >> >> >>>> >> scenario 1:
> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
> >> >> >>>> >> fine.
> >> >> >>>> >>
> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
> >> >> >>>> >> show
> >> >> >>>> >> :
> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> parent
> >> >> >>>> >> CAS
> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client.)
> >> >> >>>> >> If i try quiesce the Aggregate Service console shows :
> >> "Quiescing
> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >> >> >>>> >>
> >> >> >>>> >> scenario 2:
> >> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
> >> >> >>>> >> fine.
> >> >> >>>> >>
> >> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
> >> >> >>>> >> show
> >> >> >>>> >> :
> >> >> >>>> >> "Jms Listener Failed." ....
> >> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The
> parent
> >> >> >>>> >> CAS
> >> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
> >> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate
> service.(not
> >> >> comes
> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate
> service.(not
> >> >> comes
> >> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >> >>>> >> If i try quiesce the Aggregate Service console shows:
> >> >> >>>> >> "Quiescing
> >> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <
> uimaee@gmail.com
> >> >:
> >> >> >>>> >> > When a parent CAS comes back (after broker is restored),
> can
> >> >> >>>> >> > the
> >> >> >>>> client
> >> >> >>>> >> > send more CASes to the service and receive responses? I am
> >> >> >>>> >> > trying
> >> >> >>>> >> > to
> >> >> >>>> >> > determine what the real problem is here.
> >> >> >>>> >> >
> >> >> >>>> >> > 1) JMS Listener not recovering connection
> >> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
> >> >> >>>> connection is
> >> >> >>>> >> > stale
> >> >> >>>> >> >
> >> >> >>>> >> > -jerry
> >> >> >>>> >> >
> >> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
> >> >> >>>> uimaee@gmail.com>
> >> >> >>>> >> > wrote:
> >> >> >>>> >> >
> >> >> >>>> >> >> Strange. You say that the parent CAS comes back but the
> >> >> >>>> >> >> child
> >> >> CAS
> >> >> >>>> >> doesnt.
> >> >> >>>> >> >> Are there any exceptions in the service log? Perhaps the
> >> >> >>>> >> >> service
> >> >> >>>> >> >> is
> >> >> >>>> >> >> unable
> >> >> >>>> >> >> to send the child CAS due to missing broker. There should
> >> >> >>>> >> >> be
> >> >> >>>> something
> >> >> >>>> >> in
> >> >> >>>> >> >> the uima-as log about this.
> >> >> >>>> >> >>
> >> >> >>>> >> >> -jerry
> >> >> >>>> >> >>
> >> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >> >>>> >> >> <ne...@gmail.com>
> >> >> >>>> >> >> wrote:
> >> >> >>>> >> >>
> >> >> >>>> >> >>> Yes, it is supposed to do the service, given a
> >> >> >>>> >> >>> CAS it produces a child CAS and and this CAS continue in
> >> >> >>>> >> >>> the
> >> >> >>>> >> >>> flow,
> >> >> >>>> >> >>> that are other primitive annotators. And I expected that
> >> >> >>>> >> >>> the
> >> >> >>>> >> >>> child
> >> >> >>>> >> >>> CAS
> >> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> the scenario:
> >> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to
> >> >> >>>> >> >>> verify
> >> >> >>>> >> >>> everything is fine. The process is correct.
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS
> >> log
> >> >> >>>> show :
> >> >> >>>> >> >>> "Jms Listener Failed." ....
> >> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> Thank for you quick reply
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
> >> uimaee@gmail.com
> >> >> >:
> >> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
> >> Multiplier.
> >> >> >>>> Meaning
> >> >> >>>> >> >>> given a
> >> >> >>>> >> >>> > CAS it produces child CASes
> >> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what
> this
> >> >> >>>> service is
> >> >> >>>> >> >>> > supposed to do if the broker is running?
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> > Can you confirm the scenario:
> >> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >> >> >>>> >> >>> > 2) Service receives the CAS
> >> >> >>>> >> >>> > 3) Broker dies
> >> >> >>>> >> >>> > 4) Broker is restarted
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply
> >> >> >>>> >> >>> > when
> >> >> a
> >> >> >>>> >> >>> > broker
> >> >> >>>> >> >>> > is
> >> >> >>>> >> >>> > restarted?
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> > -Jerry
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >> >>>> >> >>> nelsonrivera12@gmail.com>
> >> >> >>>> >> >>> > wrote:
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>> >> I will try the variants that you say. I was deepening
> >> >> >>>> >> >>> >> the
> >> >> >>>> problem:
> >> >> >>>> >> >>> >>
> >> >> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
> >> >> >>>> >> >>> >> original
> >> >> >>>> >> >>> >> CAS
> >> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
> >> >> >>>> generates
> >> >> >>>> >> >>> >> new
> >> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
> >> >> >>>> >> >>> >> Aggregate.
> >> >> >>>> >> >>> >>
> >> >> >>>> >> >>> >> After restart broker (without restart the service of
> >> >> >>>> >> >>> >> annotator)
> >> >> >>>> >> >>> >> and
> >> >> >>>> >> >>> >> send a CAS to process, the listener associated to the
> >> >> client
> >> >> >>>> api
> >> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified
> >> >> >>>> >> >>> >> when
> >> >> >>>> the
> >> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in
> >> >> >>>> >> >>> >> the
> >> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >> >> >>>> >> >>> >>
> >> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but how
> >> >> >>>> >> >>> >> the
> >> >> >>>> original
> >> >> >>>> >> >>> >> CAS
> >> >> >>>> >> >>> >> is notifies into the listener when finalizes, not
> have
> >> >> >>>> >> >>> >> problem
> >> >> >>>> >> >>> >> because is not expected for another new CAS.
> >> >> >>>> >> >>> >>
> >> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
> >> >> >>>> >> >>> >> <uimaee@gmail.com
> >> >> >>>> >:
> >> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and
> >> use
> >> >> >>>> >> >>> >> > that
> >> >> >>>> >> >>> >> > broker
> >> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with?
> >> >> >>>> >> >>> >> > There
> >> >> >>>> >> >>> >> > was
> >> >> >>>> >> >>> >> > a
> >> >> >>>> >> >>> >> > nasty
> >> >> >>>> >> >>> bug
> >> >> >>>> >> >>> >> > in
> >> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am
> >> not
> >> >> >>>> sure if
> >> >> >>>> >> >>> >> > this
> >> >> >>>> >> >>> >> > is
> >> >> >>>> >> >>> >> > the cause but worth trying.
> >> >> >>>> >> >>> >> >
> >> >> >>>> >> >>> >> > You can download AMQ from here:
> >> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
> >> >> >>>> >> >>> >> >
> >> >> >>>> >> >>> >> > -Jerry
> >> >> >>>> >> >>> >> >
> >> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >> >> >>>> >> >>> >> > <ui...@gmail.com>
> >> >> >>>> >> >>> >> wrote:
> >> >> >>>> >> >>> >> >
> >> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage this
> >> >> >>>> >> >>> >> >> with
> >> >> >>>> >> >>> >> >> UIMA-AS
> >> >> >>>> >> >>> >> >> 2.8.1.
> >> >> >>>> >> >>> >> I
> >> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code. The
> >> most
> >> >> >>>> current
> >> >> >>>> >> >>> >> >> code
> >> >> >>>> >> >>> >> does
> >> >> >>>> >> >>> >> >> have newer broker.
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> I've started a broker and a sample service:
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> After the service started, I stopped the broker,
> >> waited
> >> >> >>>> >> >>> >> >> a
> >> >> >>>> bit,
> >> >> >>>> >> and
> >> >> >>>> >> >>> >> >> restarted the broker.
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the
> >> log:
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
> >> activemq.
> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >> >>>> >> >>> >> >> Controller:
> >> >> >>>> >> >>> >> >> Meeting
> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
> >> activemq.
> >> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >> >>>> >> >>> >> >> Controller:
> >> >> >>>> >> >>> >> >> Meeting
> >> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >> >>>> >> >>> >> >> Selector: Command=2001 Broker:
> tcp://localhost:61616
> >> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> >> WARNING:
> >> >> >>>> >> Service:
> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> >> WARNING:
> >> >> >>>> Jms
> >> >> >>>> >> >>> Listener
> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
> >> >> >>>> >> >>> >> >> By:
> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> java.io.EOFException
> >> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >>>> >> >>> >> WARNING:
> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> Unable
> >> To
> >> >> >>>> Connect
> >> >> >>>> >> To
> >> >> >>>> >> >>> >> >> Broker:
> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> >> WARNING:
> >> >> >>>> >> Service:
> >> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> >> WARNING:
> >> >> >>>> Jms
> >> >> >>>> >> >>> Listener
> >> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
> >> >> >>>> >> >>> >> >> By:
> >> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException:
> java.io.EOFException
> >> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >>>> >> >>> >> WARNING:
> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> Unable
> >> To
> >> >> >>>> Connect
> >> >> >>>> >> To
> >> >> >>>> >> >>> >> >> Broker:
> >> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >>>> >> >>> >> WARNING:
> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> Recovered
> >> >> >>>> >> Connection
> >> >> >>>> >> >>> to
> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
> >> >> >>>> >> >>> >> >> Again
> >> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
> >> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >> >>>> ainer.handleListenerSetupFailure:
> >> >> >>>> >> >>> >> WARNING:
> >> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> >> Recovered
> >> >> >>>> >> Connection
> >> >> >>>> >> >>> to
> >> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
> >> >> >>>> >> >>> >> >> Again
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> -Jerry
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
> >> >> >>>> >> >>> >> >> wrote:
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
> happen?
> >> >> >>>> >> >>> >> >>>
> >> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
> >> >> >>>> uimaee@gmail.com
> >> >> >>>> >> >:
> >> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >> >>>> >> >>> >> >>> >
> >> >> >>>> >> >>> >> >>> > -Jerry
> >> >> >>>> >> >>> >> >>> >
> >> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >> >>>> >> >>> >> >>> > wrote:
> >> >> >>>> >> >>> >> >>> >
> >> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
> >> >> >>>> >> >>> >> >>> >> service,
> >> >> >>>> >> >>> >> >>> >> processing
> >> >> >>>> >> >>> >> >>> >> is
> >> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker is
> >> stopped
> >> >> >>>> >> >>> >> >>> >> and
> >> >> >>>> >> >>> >> >>> >> subsequently
> >> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below
> and
> >> >> after
> >> >> >>>> this
> >> >> >>>> >> >>> >> >>> >> the
> >> >> >>>> >> >>> >> >>> >> api
> >> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the listener
> >> when
> >> >> >>>> >> >>> >> >>> >> a
> >> >> >>>> cas
> >> >> >>>> >> >>> >> >>> >> is
> >> >> >>>> >> >>> >> >>> >> processed, any more.
> >> >> >>>> >> >>> >> >>> >> can help me?:
> >> >> >>>> >> >>> >> >>> >>
> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> Analyzer
> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >>>> >> >>> Exception
> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >>>> >> >>> javax.jms.JMSException:
> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >>>> Retrying
> >> >> >>>> >> >>> >> >>> >> ...
> >> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
> >> >> >>>> Endpoint:
> >> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> >> >> >>>> M1-33610-1474481197190-1:1:1
> >> >> >>>> >> >>> Managed
> >> >> >>>> >> >>> >> >>> >> By:
> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
> >> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
> >> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> >> >>> nerContainer.
> >> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >> >> >>>> >> >>> >> >>> >> Analyzer
> >> >> >>>> >> >>> >> >>> >> Listener
> >> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
> >> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >> >>>> >> >>> >> >>> >>
> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> Analyzer
> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >>>> >> >>> Exception
> >> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException:
> >> The
> >> >> >>>> >> connection
> >> >> >>>> >> >>> is
> >> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster
> Analyzer
> >> >> >>>> >> >>> >> >>> >> Runtime
> >> >> >>>> >> >>> Exception
> >> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >> >>>> tivemq.UimaDefaultMessageListe
> >> >> >>>> >> >>> nerCont
> >> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer
> is
> >> >> >>>> closed.
> >> >> >>>> >> >>> >> >>> >>
> >> >> >>>> >> >>> >> >>> >
> >> >> >>>> >> >>> >> >>>
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >>
> >> >> >>>> >> >>> >> >
> >> >> >>>> >> >>> >>
> >> >> >>>> >> >>> >
> >> >> >>>> >> >>>
> >> >> >>>> >> >>
> >> >> >>>> >> >>
> >> >> >>>> >> >
> >> >> >>>> >>
> >> >> >>>> >
> >> >> >>>>
> >> >> >>>
> >> >> >>>
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
ok, only show you the result of last test.
Now in my test with uima-as-2.9.0, after complete restart of the
Broker UIMA-AS log show
09:51:01.731 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
Connection to Broker: tcp://localhost:61616 - Ready to Process Again
09:51:01.740 - 46:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
Connection to Broker: tcp://localhost:61616 - Ready to Process Again
09:51:01.747 - 15:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
Connection to Broker: tcp://localhost:61616 - Ready to Process Again
09:51:01.773 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
09:51:01.773 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
09:51:01.774 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException
09:51:01.774 - 14:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
09:51:01.777 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
09:51:01.777 - 19:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint:
ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
Cannot use a Temporary destination from another Connection.
this lines are continue to repeat in the log indefinitely. And after
this have the same problem when send input Cas at the service
2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Nelson, I will investigate your use case and let you know.
>
> -jerry
>
> On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> Thank you for describe how UIMA-AS works, it is very helpful. But my
>> scenario is not restart the broker while the uima-as service works on
>> input CAS
>>
>> my scenario is
>> 1- Start Broker and UIMA-AS aggregate service
>>
>> 2- Complete Restart of the Broker (without restart the service).
>> UIMA-AS log show the lines that previously i said you:
>> "Jms Listener Failed." ....
>> "The Session is closed." ...
>> "The Consumer is closed." ...
>>
>> I do not see in my uima-as log the lines that you say in your test
>> after broker is bounced:
>> Listeners reconnect:
>> INFO | Successfully refreshed JMS Connection
>> INFO | Successfully refreshed JMS Connection
>> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
>> the connection to the broker after restart.
>>
>> 4- Then in this point, after the broker was restarted completely,
>> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
>> petition UIMA-AS client receive the parent CAS, but not child CAS.
>>
>> In this moment i will trys with uima-as-2.9.0 snapshot
>>
>>
>> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Nelson, in my scenario I do have a Cas Multiplier which produces a
>> > child
>> > CAS for each input CAS. For the purpose of this
>> > test its not important how many internal delegates there are in a
>> pipeline.
>> > The service is configured as a Cas Multiplier
>> > and it does send child CASes when a broker is running.
>> >
>> > The problem is that once the input CAS is received, its reply queue is
>> > bound to a specific temp reply queue. When a broker is
>> > bounced, that queue is not valid anymore and cant be used to return
>> > child
>> > CASes nor the input CAS. To guard against such
>> > cases you need to use timeouts on the client side.
>> >
>> > Like I said in my previous email I dont get how an input CAS can be
>> > returned back to the client after a broker bounce. This should
>> > not happen since its temp queue does not exist in a new broker.
>> >
>> > I am still looking at this scenario as I don't like what I see in the
>> > service log. I think there might be a bug in the UIMA-AS but I am
>> > still investigating. Bug or not, what I described above is how UIMA-AS
>> > works. You should not expect any replies from a service
>> > working on a CAS that was sent through a broker that is bounced before
>> > a
>> > reply.
>> >
>> > -jerry
>> >
>> >
>> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera
>> > <nelsonrivera12@gmail.com
>> >
>> > wrote:
>> >
>> >> My service is a aggregate analysis engine that receives input CAS
>> >> with the direcction of a directory of documents, the first component
>> >> is a MultiplerCas that for each document generate a child CAS, each
>> >> child CAS continues the flow that are other primitive annotators
>> >> (primitive annotators in c++ that can throw exceptions), then exists
>> >> other MultiplerCas that acts as Merger, waiting the complete
>> >> processing of each child CAS and finally generate a child CAS where
>> >> each child CAS previously processed is a view of CAS, this a child CAS
>> >> that is composed of view continúa el flujo a otro AE, que espera un
>> >> CAS de esta manera.
>> >> I will send you my uima-as descriptor, to understand better my
>> >> service. All delegates are co-located.
>> >>
>> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
>> >> behaves
>> >>
>> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > I was able to stage this scenario in which an aggregate service cas
>> >> > multiplier receives a single input CAS and
>> >> > produces a child CAS.
>> >> >
>> >> > Scenario:
>> >> > 1) Service receives input CAS
>> >> > 2) CasMultiplier generates a child CAS
>> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
>> >> > 4) Broker is shutdown
>> >> > 5) AE.process() awakes from sleep
>> >> > 6) Uima-AS tries to deliver child CAS to a remote client
>> >> > 7) Connection is invalid so the child CAS is dropped
>> >> > 8) Broker is restarted
>> >> > 9) Listeners reconnect:
>> >> > INFO | Successfully refreshed JMS Connection
>> >> > INFO | Successfully refreshed JMS Connection
>> >> > 10) Broker log shows:
>> >> > WARN Service - Async error occurred:
>> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> > destination
>> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
>> >> > WARN Service - Async error occurred:
>> >> > org.apache.activemq.DestinationDoesNotExistException: The
>> >> > destination
>> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
>> >> >
>> >> > The service failed to send a reply.Nothing is delivered to a client
>> >> > as
>> >> > expected. This is the confusing part since you see a parent CAS
>> >> > being
>> >> sent
>> >> > to a client. I just cant imagine how this is possible.
>> >> >
>> >> > When a service receives an input CAS in a message, it extracts a JMS
>> >> > replyTo object which is a JMS temp queue and stores in an internal
>> >> > cache
>> >> > until it is time to send a reply. Temp queues are transient and
>> >> > unique
>> >> each
>> >> > time one is created.
>> >> >
>> >> > If the broker dies and subsequently restarts, the cached replyTo
>> >> > queue
>> >> > object is invalid (temp queue no longer exist) which leads to
>> >> > "Async error occurred:
>> >> > org.apache.activemq.DestinationDoesNotExistException" as shows
>> >> > above.
>> >> >
>> >> > In the service log I see a bunch of these exceptions:
>> >> >
>> >> > 02:05:46.574 - 56:
>> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> >> openChannel:
>> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To
>> >> > Broker:
>> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
>> >> > Retrying ...
>> >> > 02:05:46.574 - 56:
>> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> >> openChannel:
>> >> > WARNING:
>> >> > javax.jms.JMSException: Could not connect to broker URL:
>> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
>> >> > java.net.ConnectException: Connection refused
>> >> >
>> >> > This is *after* the service is restarted so there seems to be a
>> problem
>> >> > with reestablishing the connection to a new broker.
>> >> > I am still investigating that part. I just am confused how you are
>> able
>> >> to
>> >> > see a reply in the client.
>> >> >
>> >> > -Jerry
>> >> >
>> >> >
>> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> From what I understand there are two issues:
>> >> >>
>> >> >> 1) Why service JMS listeners are not reconnecting when a broker is
>> >> >> bounced. This
>> >> >> should be done by default. The 2.9.0 snapshot may address this
>> >> >> since
>> >> >> a
>> >> >> listener
>> >> >> uses newer AMQ jars (5.14.0)
>> >> >>
>> >> >> 2) What happens to a child CAS when a Broker disappears and the
>> >> >> service
>> >> >> wants
>> >> >> to deliver it to a client that is not reachable. I need to
>> >> >> investigate
>> >> >> this part. I think
>> >> >> such CAS is dropped but that should be reflected in a log and
>> also
>> >> >> the
>> >> >> input CAS
>> >> >> should be marked as failed. Again something for me to
>> investigate.
>> >> >>
>> >> >> I will try to set up a test to see if I can create the scenario.It
>> >> >> will
>> >> >> take me a bit of time
>> >> >> to set this up but I will let you know what happens.
>> >> >>
>> >> >> -jerry
>> >> >>
>> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik
>> >> >> <uimaee@gmail.com
>> >
>> >> >> wrote:
>> >> >>
>> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> >> >>> https://repository.apache.org/content/repositories/snapshots
>> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> >> >>> 23.195233-6.tar.gz
>> >> >>>
>> >> >>> Can you briefly describe your service? Does your service have any
>> >> remote
>> >> >>> delegates or all parts of the pipeline are in the same process?
>> >> >>> Do you see and exceptions in the service log?
>> >> >>>
>> >> >>> -jerry
>> >> >>>
>> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>> >> >>> <ne...@gmail.com>
>> >> >>> wrote:
>> >> >>>
>> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same error.
>> >> >>>> not
>> >> >>>> appears to be broker
>> >> >>>> issues, appears to be uima-as issues. Any other idea, i am
>> >> >>>> stagnant
>> >> >>>> at
>> >> >>>> work
>> >> >>>>
>> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is related to
>> >> broker
>> >> >>>> > issues.
>> >> >>>> >
>> >> >>>> > -jerry
>> >> >>>> >
>> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> >> >>>> nelsonrivera12@gmail.com>
>> >> >>>> > wrote:
>> >> >>>> >
>> >> >>>> >> just like that, the parent CAS comes back but the child CAS
>> >> >>>> >> doesnt.
>> >> >>>> >>
>> >> >>>> >> And the child CAS if processed, because the uima-as log
>> >> >>>> >> shows
>> >> >>>> >> the
>> >> >>>> >> lines of the other annotators primitive of the flow processing
>> >> >>>> >> the
>> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
>> >> >>>> >>
>> >> >>>> >> uima-as log only shows
>> >> >>>> >> "Jms Listener Failed. Endpoint:
>> >> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1
>> >> >>>> >> Managed
>> >> >>>> By:
>> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >>>> >> java.io.EOFException" ........
>> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>>> >> XClusterAnalyzerAggregate
>> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
>> >> >>>> >> closed..............
>> >> >>>> >>
>> >> >>>> >> The CAS is send to service after restart the Broker, and how
>> >> >>>> >> say
>> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i try
>> >> quiesce
>> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
>> Service.
>> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that is
>> >> >>>> >> the
>> >> >>>> >> child
>> >> >>>> >> CAS.
>> >> >>>> >>
>> >> >>>> >> the client if can send more CASes to the service, but not
>> receive
>> >> >>>> >> response, for each CAS send after the one CAS that was send
>> after
>> >> >>>> >> restart broker (for which parent CAS comes back but the child
>> >> >>>> >> CAS
>> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
>> >> >>>> >>
>> >> >>>> >> More specific:
>> >> >>>> >> scenario 1:
>> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
>> >> >>>> >> fine.
>> >> >>>> >>
>> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
>> >> >>>> >> show
>> >> >>>> >> :
>> >> >>>> >> "Jms Listener Failed." ....
>> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
>> >> >>>> >> CAS
>> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client.)
>> >> >>>> >> If i try quiesce the Aggregate Service console shows :
>> "Quiescing
>> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >> >>>> >>
>> >> >>>> >> scenario 2:
>> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
>> >> >>>> >> fine.
>> >> >>>> >>
>> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
>> >> >>>> >> show
>> >> >>>> >> :
>> >> >>>> >> "Jms Listener Failed." ....
>> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
>> >> >>>> >> CAS
>> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
>> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not
>> >> comes
>> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not
>> >> comes
>> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >> >>>> >> If i try quiesce the Aggregate Service console shows:
>> >> >>>> >> "Quiescing
>> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>> >:
>> >> >>>> >> > When a parent CAS comes back (after broker is restored), can
>> >> >>>> >> > the
>> >> >>>> client
>> >> >>>> >> > send more CASes to the service and receive responses? I am
>> >> >>>> >> > trying
>> >> >>>> >> > to
>> >> >>>> >> > determine what the real problem is here.
>> >> >>>> >> >
>> >> >>>> >> > 1) JMS Listener not recovering connection
>> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
>> >> >>>> connection is
>> >> >>>> >> > stale
>> >> >>>> >> >
>> >> >>>> >> > -jerry
>> >> >>>> >> >
>> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>> >> >>>> uimaee@gmail.com>
>> >> >>>> >> > wrote:
>> >> >>>> >> >
>> >> >>>> >> >> Strange. You say that the parent CAS comes back but the
>> >> >>>> >> >> child
>> >> CAS
>> >> >>>> >> doesnt.
>> >> >>>> >> >> Are there any exceptions in the service log? Perhaps the
>> >> >>>> >> >> service
>> >> >>>> >> >> is
>> >> >>>> >> >> unable
>> >> >>>> >> >> to send the child CAS due to missing broker. There should
>> >> >>>> >> >> be
>> >> >>>> something
>> >> >>>> >> in
>> >> >>>> >> >> the uima-as log about this.
>> >> >>>> >> >>
>> >> >>>> >> >> -jerry
>> >> >>>> >> >>
>> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> >>>> >> >> <ne...@gmail.com>
>> >> >>>> >> >> wrote:
>> >> >>>> >> >>
>> >> >>>> >> >>> Yes, it is supposed to do the service, given a
>> >> >>>> >> >>> CAS it produces a child CAS and and this CAS continue in
>> >> >>>> >> >>> the
>> >> >>>> >> >>> flow,
>> >> >>>> >> >>> that are other primitive annotators. And I expected that
>> >> >>>> >> >>> the
>> >> >>>> >> >>> child
>> >> >>>> >> >>> CAS
>> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
>> >> >>>> >> >>>
>> >> >>>> >> >>> the scenario:
>> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to
>> >> >>>> >> >>> verify
>> >> >>>> >> >>> everything is fine. The process is correct.
>> >> >>>> >> >>>
>> >> >>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS
>> log
>> >> >>>> show :
>> >> >>>> >> >>> "Jms Listener Failed." ....
>> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >> >>>> >> >>>
>> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>> >> >>>> >> >>>
>> >> >>>> >> >>> Thank for you quick reply
>> >> >>>> >> >>>
>> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
>> uimaee@gmail.com
>> >> >:
>> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
>> Multiplier.
>> >> >>>> Meaning
>> >> >>>> >> >>> given a
>> >> >>>> >> >>> > CAS it produces child CASes
>> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
>> >> >>>> service is
>> >> >>>> >> >>> > supposed to do if the broker is running?
>> >> >>>> >> >>> >
>> >> >>>> >> >>> > Can you confirm the scenario:
>> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >> >>>> >> >>> > 2) Service receives the CAS
>> >> >>>> >> >>> > 3) Broker dies
>> >> >>>> >> >>> > 4) Broker is restarted
>> >> >>>> >> >>> >
>> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply
>> >> >>>> >> >>> > when
>> >> a
>> >> >>>> >> >>> > broker
>> >> >>>> >> >>> > is
>> >> >>>> >> >>> > restarted?
>> >> >>>> >> >>> >
>> >> >>>> >> >>> > -Jerry
>> >> >>>> >> >>> >
>> >> >>>> >> >>> >
>> >> >>>> >> >>> >
>> >> >>>> >> >>> >
>> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >> >>>> >> >>> nelsonrivera12@gmail.com>
>> >> >>>> >> >>> > wrote:
>> >> >>>> >> >>> >
>> >> >>>> >> >>> >> I will try the variants that you say. I was deepening
>> >> >>>> >> >>> >> the
>> >> >>>> problem:
>> >> >>>> >> >>> >>
>> >> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
>> >> >>>> >> >>> >> original
>> >> >>>> >> >>> >> CAS
>> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
>> >> >>>> generates
>> >> >>>> >> >>> >> new
>> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
>> >> >>>> >> >>> >> Aggregate.
>> >> >>>> >> >>> >>
>> >> >>>> >> >>> >> After restart broker (without restart the service of
>> >> >>>> >> >>> >> annotator)
>> >> >>>> >> >>> >> and
>> >> >>>> >> >>> >> send a CAS to process, the listener associated to the
>> >> client
>> >> >>>> api
>> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified
>> >> >>>> >> >>> >> when
>> >> >>>> the
>> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in
>> >> >>>> >> >>> >> the
>> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >> >>>> >> >>> >>
>> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but how
>> >> >>>> >> >>> >> the
>> >> >>>> original
>> >> >>>> >> >>> >> CAS
>> >> >>>> >> >>> >> is notifies into the listener when finalizes, not have
>> >> >>>> >> >>> >> problem
>> >> >>>> >> >>> >> because is not expected for another new CAS.
>> >> >>>> >> >>> >>
>> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>> >> >>>> >> >>> >> <uimaee@gmail.com
>> >> >>>> >:
>> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and
>> use
>> >> >>>> >> >>> >> > that
>> >> >>>> >> >>> >> > broker
>> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with?
>> >> >>>> >> >>> >> > There
>> >> >>>> >> >>> >> > was
>> >> >>>> >> >>> >> > a
>> >> >>>> >> >>> >> > nasty
>> >> >>>> >> >>> bug
>> >> >>>> >> >>> >> > in
>> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am
>> not
>> >> >>>> sure if
>> >> >>>> >> >>> >> > this
>> >> >>>> >> >>> >> > is
>> >> >>>> >> >>> >> > the cause but worth trying.
>> >> >>>> >> >>> >> >
>> >> >>>> >> >>> >> > You can download AMQ from here:
>> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
>> >> >>>> >> >>> >> >
>> >> >>>> >> >>> >> > -Jerry
>> >> >>>> >> >>> >> >
>> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>> >> >>>> >> >>> >> > <ui...@gmail.com>
>> >> >>>> >> >>> >> wrote:
>> >> >>>> >> >>> >> >
>> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage this
>> >> >>>> >> >>> >> >> with
>> >> >>>> >> >>> >> >> UIMA-AS
>> >> >>>> >> >>> >> >> 2.8.1.
>> >> >>>> >> >>> >> I
>> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code. The
>> most
>> >> >>>> current
>> >> >>>> >> >>> >> >> code
>> >> >>>> >> >>> >> does
>> >> >>>> >> >>> >> >> have newer broker.
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> I've started a broker and a sample service:
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> After the service started, I stopped the broker,
>> waited
>> >> >>>> >> >>> >> >> a
>> >> >>>> bit,
>> >> >>>> >> and
>> >> >>>> >> >>> >> >> restarted the broker.
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the
>> log:
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
>> activemq.
>> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >> >>>> >> >>> >> >> Controller:
>> >> >>>> >> >>> >> >> Meeting
>> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
>> activemq.
>> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >> >>>> >> >>> >> >> Controller:
>> >> >>>> >> >>> >> >> Meeting
>> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> WARNING:
>> >> >>>> >> Service:
>> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> WARNING:
>> >> >>>> Jms
>> >> >>>> >> >>> Listener
>> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
>> >> >>>> >> >>> >> >> By:
>> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >>>> ainer.handleListenerSetupFailure:
>> >> >>>> >> >>> >> WARNING:
>> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable
>> To
>> >> >>>> Connect
>> >> >>>> >> To
>> >> >>>> >> >>> >> >> Broker:
>> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> WARNING:
>> >> >>>> >> Service:
>> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> >> WARNING:
>> >> >>>> Jms
>> >> >>>> >> >>> Listener
>> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
>> >> >>>> >> >>> >> >> By:
>> >> >>>> >> >>> >> >> tcp://localhost:61616
>> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >>>> ainer.handleListenerSetupFailure:
>> >> >>>> >> >>> >> WARNING:
>> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable
>> To
>> >> >>>> Connect
>> >> >>>> >> To
>> >> >>>> >> >>> >> >> Broker:
>> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >>>> ainer.handleListenerSetupFailure:
>> >> >>>> >> >>> >> WARNING:
>> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> Recovered
>> >> >>>> >> Connection
>> >> >>>> >> >>> to
>> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
>> >> >>>> >> >>> >> >> Again
>> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
>> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >> >>>> ainer.handleListenerSetupFailure:
>> >> >>>> >> >>> >> WARNING:
>> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
>> Recovered
>> >> >>>> >> Connection
>> >> >>>> >> >>> to
>> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
>> >> >>>> >> >>> >> >> Again
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> -Jerry
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
>> >> >>>> >> >>> >> >> wrote:
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >> >>>> >> >>> >> >>>
>> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>> >> >>>> uimaee@gmail.com
>> >> >>>> >> >:
>> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >> >>>> >> >>> >> >>> >
>> >> >>>> >> >>> >> >>> > -Jerry
>> >> >>>> >> >>> >> >>> >
>> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >> >>>> >> >>> >> >>> > wrote:
>> >> >>>> >> >>> >> >>> >
>> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
>> >> >>>> >> >>> >> >>> >> service,
>> >> >>>> >> >>> >> >>> >> processing
>> >> >>>> >> >>> >> >>> >> is
>> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker is
>> stopped
>> >> >>>> >> >>> >> >>> >> and
>> >> >>>> >> >>> >> >>> >> subsequently
>> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and
>> >> after
>> >> >>>> this
>> >> >>>> >> >>> >> >>> >> the
>> >> >>>> >> >>> >> >>> >> api
>> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the listener
>> when
>> >> >>>> >> >>> >> >>> >> a
>> >> >>>> cas
>> >> >>>> >> >>> >> >>> >> is
>> >> >>>> >> >>> >> >>> >> processed, any more.
>> >> >>>> >> >>> >> >>> >> can help me?:
>> >> >>>> >> >>> >> >>> >>
>> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >> >>>> >> >>> >> >>> >> Runtime
>> >> >>>> >> >>> Exception
>> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>>> >> >>> javax.jms.JMSException:
>> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> >> >>> nerContainer.
>> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >>>> >> >>> >> >>> >> Listener
>> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker:
>> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >>>> Retrying
>> >> >>>> >> >>> >> >>> >> ...
>> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
>> >> >>>> Endpoint:
>> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> >> >>>> M1-33610-1474481197190-1:1:1
>> >> >>>> >> >>> Managed
>> >> >>>> >> >>> >> >>> >> By:
>> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
>> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
>> >> >>>> >> >>> >> >>> >> java.io.EOFException
>> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> >> >>> nerContainer.
>> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>> >> >>>> >> >>> >> >>> >> Analyzer
>> >> >>>> >> >>> >> >>> >> Listener
>> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
>> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >> >>>> >> >>> >> >>> >>
>> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >> >>>> >> >>> >> >>> >> Runtime
>> >> >>>> >> >>> Exception
>> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException:
>> The
>> >> >>>> >> connection
>> >> >>>> >> >>> is
>> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >> >>>> >> >>> >> >>> >> Runtime
>> >> >>>> >> >>> Exception
>> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >> >>>> tivemq.UimaDefaultMessageListe
>> >> >>>> >> >>> nerCont
>> >> >>>> >> >>> >> >>> >> ainer.onException:
>> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
>> >> >>>> closed.
>> >> >>>> >> >>> >> >>> >>
>> >> >>>> >> >>> >> >>> >
>> >> >>>> >> >>> >> >>>
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >>
>> >> >>>> >> >>> >> >
>> >> >>>> >> >>> >>
>> >> >>>> >> >>> >
>> >> >>>> >> >>>
>> >> >>>> >> >>
>> >> >>>> >> >>
>> >> >>>> >> >
>> >> >>>> >>
>> >> >>>> >
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Nelson, I will investigate your use case and let you know.
-jerry
On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera <ne...@gmail.com>
wrote:
> Thank you for describe how UIMA-AS works, it is very helpful. But my
> scenario is not restart the broker while the uima-as service works on
> input CAS
>
> my scenario is
> 1- Start Broker and UIMA-AS aggregate service
>
> 2- Complete Restart of the Broker (without restart the service).
> UIMA-AS log show the lines that previously i said you:
> "Jms Listener Failed." ....
> "The Session is closed." ...
> "The Consumer is closed." ...
>
> I do not see in my uima-as log the lines that you say in your test
> after broker is bounced:
> Listeners reconnect:
> INFO | Successfully refreshed JMS Connection
> INFO | Successfully refreshed JMS Connection
> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
> the connection to the broker after restart.
>
> 4- Then in this point, after the broker was restarted completely,
> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
> petition UIMA-AS client receive the parent CAS, but not child CAS.
>
> In this moment i will trys with uima-as-2.9.0 snapshot
>
>
> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Nelson, in my scenario I do have a Cas Multiplier which produces a child
> > CAS for each input CAS. For the purpose of this
> > test its not important how many internal delegates there are in a
> pipeline.
> > The service is configured as a Cas Multiplier
> > and it does send child CASes when a broker is running.
> >
> > The problem is that once the input CAS is received, its reply queue is
> > bound to a specific temp reply queue. When a broker is
> > bounced, that queue is not valid anymore and cant be used to return child
> > CASes nor the input CAS. To guard against such
> > cases you need to use timeouts on the client side.
> >
> > Like I said in my previous email I dont get how an input CAS can be
> > returned back to the client after a broker bounce. This should
> > not happen since its temp queue does not exist in a new broker.
> >
> > I am still looking at this scenario as I don't like what I see in the
> > service log. I think there might be a bug in the UIMA-AS but I am
> > still investigating. Bug or not, what I described above is how UIMA-AS
> > works. You should not expect any replies from a service
> > working on a CAS that was sent through a broker that is bounced before a
> > reply.
> >
> > -jerry
> >
> >
> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera <nelsonrivera12@gmail.com
> >
> > wrote:
> >
> >> My service is a aggregate analysis engine that receives input CAS
> >> with the direcction of a directory of documents, the first component
> >> is a MultiplerCas that for each document generate a child CAS, each
> >> child CAS continues the flow that are other primitive annotators
> >> (primitive annotators in c++ that can throw exceptions), then exists
> >> other MultiplerCas that acts as Merger, waiting the complete
> >> processing of each child CAS and finally generate a child CAS where
> >> each child CAS previously processed is a view of CAS, this a child CAS
> >> that is composed of view continúa el flujo a otro AE, que espera un
> >> CAS de esta manera.
> >> I will send you my uima-as descriptor, to understand better my
> >> service. All delegates are co-located.
> >>
> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
> >> behaves
> >>
> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > I was able to stage this scenario in which an aggregate service cas
> >> > multiplier receives a single input CAS and
> >> > produces a child CAS.
> >> >
> >> > Scenario:
> >> > 1) Service receives input CAS
> >> > 2) CasMultiplier generates a child CAS
> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> >> > 4) Broker is shutdown
> >> > 5) AE.process() awakes from sleep
> >> > 6) Uima-AS tries to deliver child CAS to a remote client
> >> > 7) Connection is invalid so the child CAS is dropped
> >> > 8) Broker is restarted
> >> > 9) Listeners reconnect:
> >> > INFO | Successfully refreshed JMS Connection
> >> > INFO | Successfully refreshed JMS Connection
> >> > 10) Broker log shows:
> >> > WARN Service - Async error occurred:
> >> > org.apache.activemq.DestinationDoesNotExistException: The destination
> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> >> > WARN Service - Async error occurred:
> >> > org.apache.activemq.DestinationDoesNotExistException: The destination
> >> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> >> >
> >> > The service failed to send a reply.Nothing is delivered to a client as
> >> > expected. This is the confusing part since you see a parent CAS being
> >> sent
> >> > to a client. I just cant imagine how this is possible.
> >> >
> >> > When a service receives an input CAS in a message, it extracts a JMS
> >> > replyTo object which is a JMS temp queue and stores in an internal
> >> > cache
> >> > until it is time to send a reply. Temp queues are transient and unique
> >> each
> >> > time one is created.
> >> >
> >> > If the broker dies and subsequently restarts, the cached replyTo queue
> >> > object is invalid (temp queue no longer exist) which leads to
> >> > "Async error occurred:
> >> > org.apache.activemq.DestinationDoesNotExistException" as shows above.
> >> >
> >> > In the service log I see a bunch of these exceptions:
> >> >
> >> > 02:05:46.574 - 56:
> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> >> openChannel:
> >> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To
> >> > Broker:
> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
> >> > Retrying ...
> >> > 02:05:46.574 - 56:
> >> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> >> openChannel:
> >> > WARNING:
> >> > javax.jms.JMSException: Could not connect to broker URL:
> >> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
> >> > java.net.ConnectException: Connection refused
> >> >
> >> > This is *after* the service is restarted so there seems to be a
> problem
> >> > with reestablishing the connection to a new broker.
> >> > I am still investigating that part. I just am confused how you are
> able
> >> to
> >> > see a reply in the client.
> >> >
> >> > -Jerry
> >> >
> >> >
> >> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> >> wrote:
> >> >
> >> >> From what I understand there are two issues:
> >> >>
> >> >> 1) Why service JMS listeners are not reconnecting when a broker is
> >> >> bounced. This
> >> >> should be done by default. The 2.9.0 snapshot may address this
> >> >> since
> >> >> a
> >> >> listener
> >> >> uses newer AMQ jars (5.14.0)
> >> >>
> >> >> 2) What happens to a child CAS when a Broker disappears and the
> >> >> service
> >> >> wants
> >> >> to deliver it to a client that is not reachable. I need to
> >> >> investigate
> >> >> this part. I think
> >> >> such CAS is dropped but that should be reflected in a log and
> also
> >> >> the
> >> >> input CAS
> >> >> should be marked as failed. Again something for me to
> investigate.
> >> >>
> >> >> I will try to set up a test to see if I can create the scenario.It
> >> >> will
> >> >> take me a bit of time
> >> >> to set this up but I will let you know what happens.
> >> >>
> >> >> -jerry
> >> >>
> >> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <uimaee@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
> >> >>> https://repository.apache.org/content/repositories/snapshots
> >> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
> >> >>> 23.195233-6.tar.gz
> >> >>>
> >> >>> Can you briefly describe your service? Does your service have any
> >> remote
> >> >>> delegates or all parts of the pipeline are in the same process?
> >> >>> Do you see and exceptions in the service log?
> >> >>>
> >> >>> -jerry
> >> >>>
> >> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
> >> >>> <ne...@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
> >> >>>> appears to be broker
> >> >>>> issues, appears to be uima-as issues. Any other idea, i am stagnant
> >> >>>> at
> >> >>>> work
> >> >>>>
> >> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >>>> > Can you install AMQ broker 5.14.0 to see if this is related to
> >> broker
> >> >>>> > issues.
> >> >>>> >
> >> >>>> > -jerry
> >> >>>> >
> >> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> >> >>>> nelsonrivera12@gmail.com>
> >> >>>> > wrote:
> >> >>>> >
> >> >>>> >> just like that, the parent CAS comes back but the child CAS
> >> >>>> >> doesnt.
> >> >>>> >>
> >> >>>> >> And the child CAS if processed, because the uima-as log shows
> >> >>>> >> the
> >> >>>> >> lines of the other annotators primitive of the flow processing
> >> >>>> >> the
> >> >>>> >> child CAS, but not comes back to UIMA-AS client.
> >> >>>> >>
> >> >>>> >> uima-as log only shows
> >> >>>> >> "Jms Listener Failed. Endpoint:
> >> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1
> >> >>>> >> Managed
> >> >>>> By:
> >> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >>>> >> java.io.EOFException" ........
> >> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>>> >> XClusterAnalyzerAggregate
> >> >>>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>>> >> javax.jms.IllegalStateException: The Consumer is
> >> >>>> >> closed..............
> >> >>>> >>
> >> >>>> >> The CAS is send to service after restart the Broker, and how say
> >> >>>> >> parent CAS comes back but the child CAS doesnt. And if i try
> >> quiesce
> >> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS
> Service.
> >> >>>> >> Remaining Number of CASes to Process:1". I suppose that is the
> >> >>>> >> child
> >> >>>> >> CAS.
> >> >>>> >>
> >> >>>> >> the client if can send more CASes to the service, but not
> receive
> >> >>>> >> response, for each CAS send after the one CAS that was send
> after
> >> >>>> >> restart broker (for which parent CAS comes back but the child
> >> >>>> >> CAS
> >> >>>> >> doesnt) not comes back parent CAS or child CAS.
> >> >>>> >>
> >> >>>> >> More specific:
> >> >>>> >> scenario 1:
> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
> >> >>>> >> fine.
> >> >>>> >>
> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
> >> >>>> >> show
> >> >>>> >> :
> >> >>>> >> "Jms Listener Failed." ....
> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
> >> >>>> >> CAS
> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client.)
> >> >>>> >> If i try quiesce the Aggregate Service console shows :
> "Quiescing
> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >> >>>> >>
> >> >>>> >> scenario 2:
> >> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
> >> >>>> >> fine.
> >> >>>> >>
> >> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
> >> >>>> >> show
> >> >>>> >> :
> >> >>>> >> "Jms Listener Failed." ....
> >> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
> >> >>>> >> CAS
> >> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
> >> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not
> >> comes
> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not
> >> comes
> >> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >> >>>> >> If i try quiesce the Aggregate Service console shows:
> >> >>>> >> "Quiescing
> >> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
> >:
> >> >>>> >> > When a parent CAS comes back (after broker is restored), can
> >> >>>> >> > the
> >> >>>> client
> >> >>>> >> > send more CASes to the service and receive responses? I am
> >> >>>> >> > trying
> >> >>>> >> > to
> >> >>>> >> > determine what the real problem is here.
> >> >>>> >> >
> >> >>>> >> > 1) JMS Listener not recovering connection
> >> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
> >> >>>> connection is
> >> >>>> >> > stale
> >> >>>> >> >
> >> >>>> >> > -jerry
> >> >>>> >> >
> >> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
> >> >>>> uimaee@gmail.com>
> >> >>>> >> > wrote:
> >> >>>> >> >
> >> >>>> >> >> Strange. You say that the parent CAS comes back but the child
> >> CAS
> >> >>>> >> doesnt.
> >> >>>> >> >> Are there any exceptions in the service log? Perhaps the
> >> >>>> >> >> service
> >> >>>> >> >> is
> >> >>>> >> >> unable
> >> >>>> >> >> to send the child CAS due to missing broker. There should be
> >> >>>> something
> >> >>>> >> in
> >> >>>> >> >> the uima-as log about this.
> >> >>>> >> >>
> >> >>>> >> >> -jerry
> >> >>>> >> >>
> >> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >>>> >> >> <ne...@gmail.com>
> >> >>>> >> >> wrote:
> >> >>>> >> >>
> >> >>>> >> >>> Yes, it is supposed to do the service, given a
> >> >>>> >> >>> CAS it produces a child CAS and and this CAS continue in the
> >> >>>> >> >>> flow,
> >> >>>> >> >>> that are other primitive annotators. And I expected that the
> >> >>>> >> >>> child
> >> >>>> >> >>> CAS
> >> >>>> >> >>> will sent to UIMA-AS client, but not do.
> >> >>>> >> >>>
> >> >>>> >> >>> the scenario:
> >> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >> >>>> >> >>> everything is fine. The process is correct.
> >> >>>> >> >>>
> >> >>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS
> log
> >> >>>> show :
> >> >>>> >> >>> "Jms Listener Failed." ....
> >> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >>>> >> >>>
> >> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >> >>>> >> >>>
> >> >>>> >> >>> Thank for you quick reply
> >> >>>> >> >>>
> >> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <
> uimaee@gmail.com
> >> >:
> >> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas
> Multiplier.
> >> >>>> Meaning
> >> >>>> >> >>> given a
> >> >>>> >> >>> > CAS it produces child CASes
> >> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
> >> >>>> service is
> >> >>>> >> >>> > supposed to do if the broker is running?
> >> >>>> >> >>> >
> >> >>>> >> >>> > Can you confirm the scenario:
> >> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >> >>>> >> >>> > 2) Service receives the CAS
> >> >>>> >> >>> > 3) Broker dies
> >> >>>> >> >>> > 4) Broker is restarted
> >> >>>> >> >>> >
> >> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply
> >> >>>> >> >>> > when
> >> a
> >> >>>> >> >>> > broker
> >> >>>> >> >>> > is
> >> >>>> >> >>> > restarted?
> >> >>>> >> >>> >
> >> >>>> >> >>> > -Jerry
> >> >>>> >> >>> >
> >> >>>> >> >>> >
> >> >>>> >> >>> >
> >> >>>> >> >>> >
> >> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >>>> >> >>> nelsonrivera12@gmail.com>
> >> >>>> >> >>> > wrote:
> >> >>>> >> >>> >
> >> >>>> >> >>> >> I will try the variants that you say. I was deepening the
> >> >>>> problem:
> >> >>>> >> >>> >>
> >> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
> >> >>>> >> >>> >> original
> >> >>>> >> >>> >> CAS
> >> >>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
> >> >>>> generates
> >> >>>> >> >>> >> new
> >> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
> >> >>>> >> >>> >> Aggregate.
> >> >>>> >> >>> >>
> >> >>>> >> >>> >> After restart broker (without restart the service of
> >> >>>> >> >>> >> annotator)
> >> >>>> >> >>> >> and
> >> >>>> >> >>> >> send a CAS to process, the listener associated to the
> >> client
> >> >>>> api
> >> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified
> >> >>>> >> >>> >> when
> >> >>>> the
> >> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in the
> >> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >> >>>> >> >>> >>
> >> >>>> >> >>> >> with Aggregate AE simple the log is the same, but how the
> >> >>>> original
> >> >>>> >> >>> >> CAS
> >> >>>> >> >>> >> is notifies into the listener when finalizes, not have
> >> >>>> >> >>> >> problem
> >> >>>> >> >>> >> because is not expected for another new CAS.
> >> >>>> >> >>> >>
> >> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
> >> >>>> >> >>> >> <uimaee@gmail.com
> >> >>>> >:
> >> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and
> use
> >> >>>> >> >>> >> > that
> >> >>>> >> >>> >> > broker
> >> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There
> >> >>>> >> >>> >> > was
> >> >>>> >> >>> >> > a
> >> >>>> >> >>> >> > nasty
> >> >>>> >> >>> bug
> >> >>>> >> >>> >> > in
> >> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am
> not
> >> >>>> sure if
> >> >>>> >> >>> >> > this
> >> >>>> >> >>> >> > is
> >> >>>> >> >>> >> > the cause but worth trying.
> >> >>>> >> >>> >> >
> >> >>>> >> >>> >> > You can download AMQ from here:
> >> >>>> >> >>> >> > http://activemq.apache.org/download.html
> >> >>>> >> >>> >> >
> >> >>>> >> >>> >> > -Jerry
> >> >>>> >> >>> >> >
> >> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >> >>>> >> >>> >> > <ui...@gmail.com>
> >> >>>> >> >>> >> wrote:
> >> >>>> >> >>> >> >
> >> >>>> >> >>> >> >> Not sure what is causing this. I need to stage this
> >> >>>> >> >>> >> >> with
> >> >>>> >> >>> >> >> UIMA-AS
> >> >>>> >> >>> >> >> 2.8.1.
> >> >>>> >> >>> >> I
> >> >>>> >> >>> >> >> am currently testing with trunk (latest) code. The
> most
> >> >>>> current
> >> >>>> >> >>> >> >> code
> >> >>>> >> >>> >> does
> >> >>>> >> >>> >> >> have newer broker.
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> I've started a broker and a sample service:
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> After the service started, I stopped the broker,
> waited
> >> >>>> >> >>> >> >> a
> >> >>>> bit,
> >> >>>> >> and
> >> >>>> >> >>> >> >> restarted the broker.
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the
> log:
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.
> activemq.
> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >>>> >> >>> >> >> Controller:
> >> >>>> >> >>> >> >> Meeting
> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.
> activemq.
> >> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >> >>>> >> >>> >> >> Controller:
> >> >>>> >> >>> >> >> Meeting
> >> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >> >>>> >> >>> >> >> 04:21:45.621 - 22:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> WARNING:
> >> >>>> >> Service:
> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> WARNING:
> >> >>>> Jms
> >> >>>> >> >>> Listener
> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>>> >> >>> >> >> 04:21:45.622 - 22:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >>>> ainer.handleListenerSetupFailure:
> >> >>>> >> >>> >> WARNING:
> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable
> To
> >> >>>> Connect
> >> >>>> >> To
> >> >>>> >> >>> >> >> Broker:
> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >>>> >> >>> >> >> 04:21:45.638 - 27:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> WARNING:
> >> >>>> >> Service:
> >> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> >> WARNING:
> >> >>>> Jms
> >> >>>> >> >>> Listener
> >> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >>>> >> >>> >> >> tcp://localhost:61616
> >> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>>> >> >>> >> >> 04:21:45.639 - 27:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >>>> ainer.handleListenerSetupFailure:
> >> >>>> >> >>> >> WARNING:
> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable
> To
> >> >>>> Connect
> >> >>>> >> To
> >> >>>> >> >>> >> >> Broker:
> >> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >>>> >> >>> >> >> 04:22:07.439 - 22:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >>>> ainer.handleListenerSetupFailure:
> >> >>>> >> >>> >> WARNING:
> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> Recovered
> >> >>>> >> Connection
> >> >>>> >> >>> to
> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >>>> >> >>> >> >> 04:22:07.454 - 27:
> >> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
> >> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >> >>>> ainer.handleListenerSetupFailure:
> >> >>>> >> >>> >> WARNING:
> >> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
> Recovered
> >> >>>> >> Connection
> >> >>>> >> >>> to
> >> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> -Jerry
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >> >>>> >> >>> >> nelsonrivera12@gmail.com>
> >> >>>> >> >>> >> >> wrote:
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >> >>>> >> >>> >> >>>
> >> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
> >> >>>> uimaee@gmail.com
> >> >>>> >> >:
> >> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >>>> >> >>> >> >>> >
> >> >>>> >> >>> >> >>> > -Jerry
> >> >>>> >> >>> >> >>> >
> >> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >>>> >> >>> >> >>> > wrote:
> >> >>>> >> >>> >> >>> >
> >> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
> >> >>>> >> >>> >> >>> >> service,
> >> >>>> >> >>> >> >>> >> processing
> >> >>>> >> >>> >> >>> >> is
> >> >>>> >> >>> >> >>> >> executed correctly, but when the broker is
> stopped
> >> >>>> >> >>> >> >>> >> and
> >> >>>> >> >>> >> >>> >> subsequently
> >> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and
> >> after
> >> >>>> this
> >> >>>> >> >>> >> >>> >> the
> >> >>>> >> >>> >> >>> >> api
> >> >>>> >> >>> >> >>> >> client uima-as is not notified in the listener
> when
> >> >>>> >> >>> >> >>> >> a
> >> >>>> cas
> >> >>>> >> >>> >> >>> >> is
> >> >>>> >> >>> >> >>> >> processed, any more.
> >> >>>> >> >>> >> >>> >> can help me?:
> >> >>>> >> >>> >> >>> >>
> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >> >>>> >> >>> >> >>> >> Runtime
> >> >>>> >> >>> Exception
> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>>> >> >>> javax.jms.JMSException:
> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> >> >>> nerContainer.
> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >> >>>> >> >>> >> >>> >> Analyzer
> >> >>>> >> >>> >> >>> >> Listener
> >> >>>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
> >> >>>> Retrying
> >> >>>> >> >>> >> >>> >> ...
> >> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
> >> >>>> Endpoint:
> >> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> >> >>>> M1-33610-1474481197190-1:1:1
> >> >>>> >> >>> Managed
> >> >>>> >> >>> >> >>> >> By:
> >> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
> >> >>>> >> >>> >> >>> >> javax.jms.JMSException:
> >> >>>> >> >>> >> >>> >> java.io.EOFException
> >> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> >> >>> nerContainer.
> >> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >> >>>> >> >>> >> >>> >> Analyzer
> >> >>>> >> >>> >> >>> >> Listener
> >> >>>> >> >>> >> >>> >> Established Connection to Broker:
> >> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >> >>>> >> >>> >> >>> >>
> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >> >>>> >> >>> >> >>> >> Runtime
> >> >>>> >> >>> Exception
> >> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException:
> The
> >> >>>> >> connection
> >> >>>> >> >>> is
> >> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >> >>>> >> >>> >> >>> >> Runtime
> >> >>>> >> >>> Exception
> >> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >> >>>> tivemq.UimaDefaultMessageListe
> >> >>>> >> >>> nerCont
> >> >>>> >> >>> >> >>> >> ainer.onException:
> >> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>>> >> >>> >> XClusterAnalyzerAggregate
> >> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
> >> >>>> closed.
> >> >>>> >> >>> >> >>> >>
> >> >>>> >> >>> >> >>> >
> >> >>>> >> >>> >> >>>
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >>
> >> >>>> >> >>> >> >
> >> >>>> >> >>> >>
> >> >>>> >> >>> >
> >> >>>> >> >>>
> >> >>>> >> >>
> >> >>>> >> >>
> >> >>>> >> >
> >> >>>> >>
> >> >>>> >
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
Thank you for describe how UIMA-AS works, it is very helpful. But my
scenario is not restart the broker while the uima-as service works on
input CAS
my scenario is
1- Start Broker and UIMA-AS aggregate service
2- Complete Restart of the Broker (without restart the service).
UIMA-AS log show the lines that previously i said you:
"Jms Listener Failed." ....
"The Session is closed." ...
"The Consumer is closed." ...
I do not see in my uima-as log the lines that you say in your test
after broker is bounced:
Listeners reconnect:
INFO | Successfully refreshed JMS Connection
INFO | Successfully refreshed JMS Connection
In my case (uima-as 2.8.1) seems to be a problem with reestablishing
the connection to the broker after restart.
4- Then in this point, after the broker was restarted completely,
uima-as client sends a CAS to UIMA-AS aggregate service. And for this
petition UIMA-AS client receive the parent CAS, but not child CAS.
In this moment i will trys with uima-as-2.9.0 snapshot
2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Nelson, in my scenario I do have a Cas Multiplier which produces a child
> CAS for each input CAS. For the purpose of this
> test its not important how many internal delegates there are in a pipeline.
> The service is configured as a Cas Multiplier
> and it does send child CASes when a broker is running.
>
> The problem is that once the input CAS is received, its reply queue is
> bound to a specific temp reply queue. When a broker is
> bounced, that queue is not valid anymore and cant be used to return child
> CASes nor the input CAS. To guard against such
> cases you need to use timeouts on the client side.
>
> Like I said in my previous email I dont get how an input CAS can be
> returned back to the client after a broker bounce. This should
> not happen since its temp queue does not exist in a new broker.
>
> I am still looking at this scenario as I don't like what I see in the
> service log. I think there might be a bug in the UIMA-AS but I am
> still investigating. Bug or not, what I described above is how UIMA-AS
> works. You should not expect any replies from a service
> working on a CAS that was sent through a broker that is bounced before a
> reply.
>
> -jerry
>
>
> On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> My service is a aggregate analysis engine that receives input CAS
>> with the direcction of a directory of documents, the first component
>> is a MultiplerCas that for each document generate a child CAS, each
>> child CAS continues the flow that are other primitive annotators
>> (primitive annotators in c++ that can throw exceptions), then exists
>> other MultiplerCas that acts as Merger, waiting the complete
>> processing of each child CAS and finally generate a child CAS where
>> each child CAS previously processed is a view of CAS, this a child CAS
>> that is composed of view continúa el flujo a otro AE, que espera un
>> CAS de esta manera.
>> I will send you my uima-as descriptor, to understand better my
>> service. All delegates are co-located.
>>
>> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
>> behaves
>>
>> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > I was able to stage this scenario in which an aggregate service cas
>> > multiplier receives a single input CAS and
>> > produces a child CAS.
>> >
>> > Scenario:
>> > 1) Service receives input CAS
>> > 2) CasMultiplier generates a child CAS
>> > 3) AE.process() is called (child CAS) and it sleeps for awhile
>> > 4) Broker is shutdown
>> > 5) AE.process() awakes from sleep
>> > 6) Uima-AS tries to deliver child CAS to a remote client
>> > 7) Connection is invalid so the child CAS is dropped
>> > 8) Broker is restarted
>> > 9) Listeners reconnect:
>> > INFO | Successfully refreshed JMS Connection
>> > INFO | Successfully refreshed JMS Connection
>> > 10) Broker log shows:
>> > WARN Service - Async error occurred:
>> > org.apache.activemq.DestinationDoesNotExistException: The destination
>> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
>> > WARN Service - Async error occurred:
>> > org.apache.activemq.DestinationDoesNotExistException: The destination
>> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
>> >
>> > The service failed to send a reply.Nothing is delivered to a client as
>> > expected. This is the confusing part since you see a parent CAS being
>> sent
>> > to a client. I just cant imagine how this is possible.
>> >
>> > When a service receives an input CAS in a message, it extracts a JMS
>> > replyTo object which is a JMS temp queue and stores in an internal
>> > cache
>> > until it is time to send a reply. Temp queues are transient and unique
>> each
>> > time one is created.
>> >
>> > If the broker dies and subsequently restarts, the cached replyTo queue
>> > object is invalid (temp queue no longer exist) which leads to
>> > "Async error occurred:
>> > org.apache.activemq.DestinationDoesNotExistException" as shows above.
>> >
>> > In the service log I see a bunch of these exceptions:
>> >
>> > 02:05:46.574 - 56:
>> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> openChannel:
>> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To
>> > Broker:
>> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
>> > Retrying ...
>> > 02:05:46.574 - 56:
>> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
>> openChannel:
>> > WARNING:
>> > javax.jms.JMSException: Could not connect to broker URL:
>> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
>> > java.net.ConnectException: Connection refused
>> >
>> > This is *after* the service is restarted so there seems to be a problem
>> > with reestablishing the connection to a new broker.
>> > I am still investigating that part. I just am confused how you are able
>> to
>> > see a reply in the client.
>> >
>> > -Jerry
>> >
>> >
>> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> wrote:
>> >
>> >> From what I understand there are two issues:
>> >>
>> >> 1) Why service JMS listeners are not reconnecting when a broker is
>> >> bounced. This
>> >> should be done by default. The 2.9.0 snapshot may address this
>> >> since
>> >> a
>> >> listener
>> >> uses newer AMQ jars (5.14.0)
>> >>
>> >> 2) What happens to a child CAS when a Broker disappears and the
>> >> service
>> >> wants
>> >> to deliver it to a client that is not reachable. I need to
>> >> investigate
>> >> this part. I think
>> >> such CAS is dropped but that should be reflected in a log and also
>> >> the
>> >> input CAS
>> >> should be marked as failed. Again something for me to investigate.
>> >>
>> >> I will try to set up a test to see if I can create the scenario.It
>> >> will
>> >> take me a bit of time
>> >> to set this up but I will let you know what happens.
>> >>
>> >> -jerry
>> >>
>> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> >> wrote:
>> >>
>> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> >>> https://repository.apache.org/content/repositories/snapshots
>> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> >>> 23.195233-6.tar.gz
>> >>>
>> >>> Can you briefly describe your service? Does your service have any
>> remote
>> >>> delegates or all parts of the pipeline are in the same process?
>> >>> Do you see and exceptions in the service log?
>> >>>
>> >>> -jerry
>> >>>
>> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>> >>> <ne...@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>> >>>> appears to be broker
>> >>>> issues, appears to be uima-as issues. Any other idea, i am stagnant
>> >>>> at
>> >>>> work
>> >>>>
>> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >>>> > Can you install AMQ broker 5.14.0 to see if this is related to
>> broker
>> >>>> > issues.
>> >>>> >
>> >>>> > -jerry
>> >>>> >
>> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> >>>> nelsonrivera12@gmail.com>
>> >>>> > wrote:
>> >>>> >
>> >>>> >> just like that, the parent CAS comes back but the child CAS
>> >>>> >> doesnt.
>> >>>> >>
>> >>>> >> And the child CAS if processed, because the uima-as log shows
>> >>>> >> the
>> >>>> >> lines of the other annotators primitive of the flow processing
>> >>>> >> the
>> >>>> >> child CAS, but not comes back to UIMA-AS client.
>> >>>> >>
>> >>>> >> uima-as log only shows
>> >>>> >> "Jms Listener Failed. Endpoint:
>> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1
>> >>>> >> Managed
>> >>>> By:
>> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >>>> >> java.io.EOFException" ........
>> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>>> >> XClusterAnalyzerAggregate
>> >>>> >> Managed By: tcp://localhost:61616 Reason:
>> >>>> >> javax.jms.IllegalStateException: The Consumer is
>> >>>> >> closed..............
>> >>>> >>
>> >>>> >> The CAS is send to service after restart the Broker, and how say
>> >>>> >> parent CAS comes back but the child CAS doesnt. And if i try
>> quiesce
>> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>> >>>> >> Remaining Number of CASes to Process:1". I suppose that is the
>> >>>> >> child
>> >>>> >> CAS.
>> >>>> >>
>> >>>> >> the client if can send more CASes to the service, but not receive
>> >>>> >> response, for each CAS send after the one CAS that was send after
>> >>>> >> restart broker (for which parent CAS comes back but the child
>> >>>> >> CAS
>> >>>> >> doesnt) not comes back parent CAS or child CAS.
>> >>>> >>
>> >>>> >> More specific:
>> >>>> >> scenario 1:
>> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
>> >>>> >> fine.
>> >>>> >>
>> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
>> >>>> >> show
>> >>>> >> :
>> >>>> >> "Jms Listener Failed." ....
>> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
>> >>>> >> CAS
>> >>>> >> comes back but the child CAS doesnt, to the uima-as client.)
>> >>>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
>> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >>>> >>
>> >>>> >> scenario 2:
>> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is
>> >>>> >> fine.
>> >>>> >>
>> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log
>> >>>> >> show
>> >>>> >> :
>> >>>> >> "Jms Listener Failed." ....
>> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent
>> >>>> >> CAS
>> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
>> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not
>> comes
>> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not
>> comes
>> >>>> >> back parent CAS or child CAS , to the uima-as client)
>> >>>> >> If i try quiesce the Aggregate Service console shows:
>> >>>> >> "Quiescing
>> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >>>> >> > When a parent CAS comes back (after broker is restored), can
>> >>>> >> > the
>> >>>> client
>> >>>> >> > send more CASes to the service and receive responses? I am
>> >>>> >> > trying
>> >>>> >> > to
>> >>>> >> > determine what the real problem is here.
>> >>>> >> >
>> >>>> >> > 1) JMS Listener not recovering connection
>> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
>> >>>> connection is
>> >>>> >> > stale
>> >>>> >> >
>> >>>> >> > -jerry
>> >>>> >> >
>> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>> >>>> uimaee@gmail.com>
>> >>>> >> > wrote:
>> >>>> >> >
>> >>>> >> >> Strange. You say that the parent CAS comes back but the child
>> CAS
>> >>>> >> doesnt.
>> >>>> >> >> Are there any exceptions in the service log? Perhaps the
>> >>>> >> >> service
>> >>>> >> >> is
>> >>>> >> >> unable
>> >>>> >> >> to send the child CAS due to missing broker. There should be
>> >>>> something
>> >>>> >> in
>> >>>> >> >> the uima-as log about this.
>> >>>> >> >>
>> >>>> >> >> -jerry
>> >>>> >> >>
>> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >>>> >> >> <ne...@gmail.com>
>> >>>> >> >> wrote:
>> >>>> >> >>
>> >>>> >> >>> Yes, it is supposed to do the service, given a
>> >>>> >> >>> CAS it produces a child CAS and and this CAS continue in the
>> >>>> >> >>> flow,
>> >>>> >> >>> that are other primitive annotators. And I expected that the
>> >>>> >> >>> child
>> >>>> >> >>> CAS
>> >>>> >> >>> will sent to UIMA-AS client, but not do.
>> >>>> >> >>>
>> >>>> >> >>> the scenario:
>> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>> >>>> >> >>> everything is fine. The process is correct.
>> >>>> >> >>>
>> >>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log
>> >>>> show :
>> >>>> >> >>> "Jms Listener Failed." ....
>> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >>>> >> >>>
>> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>> >>>> >> >>>
>> >>>> >> >>> Thank for you quick reply
>> >>>> >> >>>
>> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>> >:
>> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
>> >>>> Meaning
>> >>>> >> >>> given a
>> >>>> >> >>> > CAS it produces child CASes
>> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
>> >>>> service is
>> >>>> >> >>> > supposed to do if the broker is running?
>> >>>> >> >>> >
>> >>>> >> >>> > Can you confirm the scenario:
>> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >>>> >> >>> > 2) Service receives the CAS
>> >>>> >> >>> > 3) Broker dies
>> >>>> >> >>> > 4) Broker is restarted
>> >>>> >> >>> >
>> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply
>> >>>> >> >>> > when
>> a
>> >>>> >> >>> > broker
>> >>>> >> >>> > is
>> >>>> >> >>> > restarted?
>> >>>> >> >>> >
>> >>>> >> >>> > -Jerry
>> >>>> >> >>> >
>> >>>> >> >>> >
>> >>>> >> >>> >
>> >>>> >> >>> >
>> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >>>> >> >>> nelsonrivera12@gmail.com>
>> >>>> >> >>> > wrote:
>> >>>> >> >>> >
>> >>>> >> >>> >> I will try the variants that you say. I was deepening the
>> >>>> problem:
>> >>>> >> >>> >>
>> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
>> >>>> >> >>> >> original
>> >>>> >> >>> >> CAS
>> >>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
>> >>>> generates
>> >>>> >> >>> >> new
>> >>>> >> >>> >> CAS and is this CAS that continue in the flow of
>> >>>> >> >>> >> Aggregate.
>> >>>> >> >>> >>
>> >>>> >> >>> >> After restart broker (without restart the service of
>> >>>> >> >>> >> annotator)
>> >>>> >> >>> >> and
>> >>>> >> >>> >> send a CAS to process, the listener associated to the
>> client
>> >>>> api
>> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified
>> >>>> >> >>> >> when
>> >>>> the
>> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in the
>> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >>>> >> >>> >>
>> >>>> >> >>> >> with Aggregate AE simple the log is the same, but how the
>> >>>> original
>> >>>> >> >>> >> CAS
>> >>>> >> >>> >> is notifies into the listener when finalizes, not have
>> >>>> >> >>> >> problem
>> >>>> >> >>> >> because is not expected for another new CAS.
>> >>>> >> >>> >>
>> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>> >>>> >> >>> >> <uimaee@gmail.com
>> >>>> >:
>> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use
>> >>>> >> >>> >> > that
>> >>>> >> >>> >> > broker
>> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There
>> >>>> >> >>> >> > was
>> >>>> >> >>> >> > a
>> >>>> >> >>> >> > nasty
>> >>>> >> >>> bug
>> >>>> >> >>> >> > in
>> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am not
>> >>>> sure if
>> >>>> >> >>> >> > this
>> >>>> >> >>> >> > is
>> >>>> >> >>> >> > the cause but worth trying.
>> >>>> >> >>> >> >
>> >>>> >> >>> >> > You can download AMQ from here:
>> >>>> >> >>> >> > http://activemq.apache.org/download.html
>> >>>> >> >>> >> >
>> >>>> >> >>> >> > -Jerry
>> >>>> >> >>> >> >
>> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>> >>>> >> >>> >> > <ui...@gmail.com>
>> >>>> >> >>> >> wrote:
>> >>>> >> >>> >> >
>> >>>> >> >>> >> >> Not sure what is causing this. I need to stage this
>> >>>> >> >>> >> >> with
>> >>>> >> >>> >> >> UIMA-AS
>> >>>> >> >>> >> >> 2.8.1.
>> >>>> >> >>> >> I
>> >>>> >> >>> >> >> am currently testing with trunk (latest) code. The most
>> >>>> current
>> >>>> >> >>> >> >> code
>> >>>> >> >>> >> does
>> >>>> >> >>> >> >> have newer broker.
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> I've started a broker and a sample service:
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> After the service started, I stopped the broker, waited
>> >>>> >> >>> >> >> a
>> >>>> bit,
>> >>>> >> and
>> >>>> >> >>> >> >> restarted the broker.
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >>>> >> >>> >> >> Controller:
>> >>>> >> >>> >> >> Meeting
>> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >>>> >> >>> >> >> tcp://localhost:61616
>> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>> >>>> >> >>> >> >> Controller:
>> >>>> >> >>> >> >> Meeting
>> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >>>> >> >>> >> >> 04:21:45.621 - 22:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> WARNING:
>> >>>> >> Service:
>> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> WARNING:
>> >>>> Jms
>> >>>> >> >>> Listener
>> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >>>> >> >>> >> >> tcp://localhost:61616
>> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >>>> >> >>> >> >> 04:21:45.622 - 22:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >>>> ainer.handleListenerSetupFailure:
>> >>>> >> >>> >> WARNING:
>> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>> >>>> Connect
>> >>>> >> To
>> >>>> >> >>> >> >> Broker:
>> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >>>> >> >>> >> >> 04:21:45.638 - 27:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> WARNING:
>> >>>> >> Service:
>> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
>> WARNING:
>> >>>> Jms
>> >>>> >> >>> Listener
>> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >>>> >> >>> >> >> tcp://localhost:61616
>> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >>>> >> >>> >> >> 04:21:45.639 - 27:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >>>> ainer.handleListenerSetupFailure:
>> >>>> >> >>> >> WARNING:
>> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>> >>>> Connect
>> >>>> >> To
>> >>>> >> >>> >> >> Broker:
>> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >>>> >> >>> >> >> 04:22:07.439 - 22:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >>>> ainer.handleListenerSetupFailure:
>> >>>> >> >>> >> WARNING:
>> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> >>>> >> Connection
>> >>>> >> >>> to
>> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >>>> >> >>> >> >> 04:22:07.454 - 27:
>> >>>> >> >>> >> >> org.apache.uima.adapter.jms.activemq.
>> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
>> >>>> ainer.handleListenerSetupFailure:
>> >>>> >> >>> >> WARNING:
>> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> >>>> >> Connection
>> >>>> >> >>> to
>> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> -Jerry
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >>>> >> >>> >> nelsonrivera12@gmail.com>
>> >>>> >> >>> >> >> wrote:
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >>>> >> >>> >> >>>
>> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>> >>>> uimaee@gmail.com
>> >>>> >> >:
>> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >>>> >> >>> >> >>> >
>> >>>> >> >>> >> >>> > -Jerry
>> >>>> >> >>> >> >>> >
>> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >>>> >> >>> >> >>> > wrote:
>> >>>> >> >>> >> >>> >
>> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
>> >>>> >> >>> >> >>> >> service,
>> >>>> >> >>> >> >>> >> processing
>> >>>> >> >>> >> >>> >> is
>> >>>> >> >>> >> >>> >> executed correctly, but when the broker is stopped
>> >>>> >> >>> >> >>> >> and
>> >>>> >> >>> >> >>> >> subsequently
>> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and
>> after
>> >>>> this
>> >>>> >> >>> >> >>> >> the
>> >>>> >> >>> >> >>> >> api
>> >>>> >> >>> >> >>> >> client uima-as is not notified in the listener when
>> >>>> >> >>> >> >>> >> a
>> >>>> cas
>> >>>> >> >>> >> >>> >> is
>> >>>> >> >>> >> >>> >> processed, any more.
>> >>>> >> >>> >> >>> >> can help me?:
>> >>>> >> >>> >> >>> >>
>> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >>>> >> >>> >> >>> >> Runtime
>> >>>> >> >>> Exception
>> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>>> >> >>> javax.jms.JMSException:
>> >>>> >> >>> >> >>> >> java.io.EOFException
>> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> >> >>> nerContainer.
>> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>> >>>> >> >>> >> >>> >> Analyzer
>> >>>> >> >>> >> >>> >> Listener
>> >>>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
>> >>>> Retrying
>> >>>> >> >>> >> >>> >> ...
>> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
>> >>>> Endpoint:
>> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> >>>> M1-33610-1474481197190-1:1:1
>> >>>> >> >>> Managed
>> >>>> >> >>> >> >>> >> By:
>> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason:
>> >>>> >> >>> >> >>> >> javax.jms.JMSException:
>> >>>> >> >>> >> >>> >> java.io.EOFException
>> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> >> >>> nerContainer.
>> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>> >>>> >> >>> >> >>> >> Analyzer
>> >>>> >> >>> >> >>> >> Listener
>> >>>> >> >>> >> >>> >> Established Connection to Broker:
>> >>>> >> >>> >> >>> >> tcp://localhost:61616
>> >>>> >> >>> >> >>> >>
>> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >>>> >> >>> >> >>> >> Runtime
>> >>>> >> >>> Exception
>> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
>> >>>> >> connection
>> >>>> >> >>> is
>> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>> >>>> >> >>> >> >>> >> Runtime
>> >>>> >> >>> Exception
>> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> >>>> tivemq.UimaDefaultMessageListe
>> >>>> >> >>> nerCont
>> >>>> >> >>> >> >>> >> ainer.onException:
>> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>>> >> >>> >> XClusterAnalyzerAggregate
>> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
>> >>>> closed.
>> >>>> >> >>> >> >>> >>
>> >>>> >> >>> >> >>> >
>> >>>> >> >>> >> >>>
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >>
>> >>>> >> >>> >> >
>> >>>> >> >>> >>
>> >>>> >> >>> >
>> >>>> >> >>>
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >
>> >>>> >>
>> >>>> >
>> >>>>
>> >>>
>> >>>
>> >>
>> >
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Nelson, in my scenario I do have a Cas Multiplier which produces a child
CAS for each input CAS. For the purpose of this
test its not important how many internal delegates there are in a pipeline.
The service is configured as a Cas Multiplier
and it does send child CASes when a broker is running.
The problem is that once the input CAS is received, its reply queue is
bound to a specific temp reply queue. When a broker is
bounced, that queue is not valid anymore and cant be used to return child
CASes nor the input CAS. To guard against such
cases you need to use timeouts on the client side.
Like I said in my previous email I dont get how an input CAS can be
returned back to the client after a broker bounce. This should
not happen since its temp queue does not exist in a new broker.
I am still looking at this scenario as I don't like what I see in the
service log. I think there might be a bug in the UIMA-AS but I am
still investigating. Bug or not, what I described above is how UIMA-AS
works. You should not expect any replies from a service
working on a CAS that was sent through a broker that is bounced before a
reply.
-jerry
On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera <ne...@gmail.com>
wrote:
> My service is a aggregate analysis engine that receives input CAS
> with the direcction of a directory of documents, the first component
> is a MultiplerCas that for each document generate a child CAS, each
> child CAS continues the flow that are other primitive annotators
> (primitive annotators in c++ that can throw exceptions), then exists
> other MultiplerCas that acts as Merger, waiting the complete
> processing of each child CAS and finally generate a child CAS where
> each child CAS previously processed is a view of CAS, this a child CAS
> that is composed of view continúa el flujo a otro AE, que espera un
> CAS de esta manera.
> I will send you my uima-as descriptor, to understand better my
> service. All delegates are co-located.
>
> i will trys with uima-as-2.9.0 snapshot how you say, to see how it behaves
>
> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > I was able to stage this scenario in which an aggregate service cas
> > multiplier receives a single input CAS and
> > produces a child CAS.
> >
> > Scenario:
> > 1) Service receives input CAS
> > 2) CasMultiplier generates a child CAS
> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> > 4) Broker is shutdown
> > 5) AE.process() awakes from sleep
> > 6) Uima-AS tries to deliver child CAS to a remote client
> > 7) Connection is invalid so the child CAS is dropped
> > 8) Broker is restarted
> > 9) Listeners reconnect:
> > INFO | Successfully refreshed JMS Connection
> > INFO | Successfully refreshed JMS Connection
> > 10) Broker log shows:
> > WARN Service - Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException: The destination
> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> > WARN Service - Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException: The destination
> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> >
> > The service failed to send a reply.Nothing is delivered to a client as
> > expected. This is the confusing part since you see a parent CAS being
> sent
> > to a client. I just cant imagine how this is possible.
> >
> > When a service receives an input CAS in a message, it extracts a JMS
> > replyTo object which is a JMS temp queue and stores in an internal cache
> > until it is time to send a reply. Temp queues are transient and unique
> each
> > time one is created.
> >
> > If the broker dies and subsequently restarts, the cached replyTo queue
> > object is invalid (temp queue no longer exist) which leads to
> > "Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException" as shows above.
> >
> > In the service log I see a bunch of these exceptions:
> >
> > 02:05:46.574 - 56:
> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> openChannel:
> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To Broker:
> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
> > Retrying ...
> > 02:05:46.574 - 56:
> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> openChannel:
> > WARNING:
> > javax.jms.JMSException: Could not connect to broker URL:
> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
> > java.net.ConnectException: Connection refused
> >
> > This is *after* the service is restarted so there seems to be a problem
> > with reestablishing the connection to a new broker.
> > I am still investigating that part. I just am confused how you are able
> to
> > see a reply in the client.
> >
> > -Jerry
> >
> >
> > On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> wrote:
> >
> >> From what I understand there are two issues:
> >>
> >> 1) Why service JMS listeners are not reconnecting when a broker is
> >> bounced. This
> >> should be done by default. The 2.9.0 snapshot may address this since
> >> a
> >> listener
> >> uses newer AMQ jars (5.14.0)
> >>
> >> 2) What happens to a child CAS when a Broker disappears and the service
> >> wants
> >> to deliver it to a client that is not reachable. I need to
> >> investigate
> >> this part. I think
> >> such CAS is dropped but that should be reflected in a log and also
> >> the
> >> input CAS
> >> should be marked as failed. Again something for me to investigate.
> >>
> >> I will try to set up a test to see if I can create the scenario.It will
> >> take me a bit of time
> >> to set this up but I will let you know what happens.
> >>
> >> -jerry
> >>
> >> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> >> wrote:
> >>
> >>> Here is the uima-as 2.9.0-SNAPSHOT to try:
> >>> https://repository.apache.org/content/repositories/snapshots
> >>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
> >>> 23.195233-6.tar.gz
> >>>
> >>> Can you briefly describe your service? Does your service have any
> remote
> >>> delegates or all parts of the pipeline are in the same process?
> >>> Do you see and exceptions in the service log?
> >>>
> >>> -jerry
> >>>
> >>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
> >>> <ne...@gmail.com>
> >>> wrote:
> >>>
> >>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
> >>>> appears to be broker
> >>>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
> >>>> work
> >>>>
> >>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >>>> > Can you install AMQ broker 5.14.0 to see if this is related to
> broker
> >>>> > issues.
> >>>> >
> >>>> > -jerry
> >>>> >
> >>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> >>>> nelsonrivera12@gmail.com>
> >>>> > wrote:
> >>>> >
> >>>> >> just like that, the parent CAS comes back but the child CAS doesnt.
> >>>> >>
> >>>> >> And the child CAS if processed, because the uima-as log shows the
> >>>> >> lines of the other annotators primitive of the flow processing the
> >>>> >> child CAS, but not comes back to UIMA-AS client.
> >>>> >>
> >>>> >> uima-as log only shows
> >>>> >> "Jms Listener Failed. Endpoint:
> >>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed
> >>>> By:
> >>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >>>> >> java.io.EOFException" ........
> >>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>>> >> XClusterAnalyzerAggregate
> >>>> >> Managed By: tcp://localhost:61616 Reason:
> >>>> >> javax.jms.IllegalStateException: The Consumer is
> >>>> >> closed..............
> >>>> >>
> >>>> >> The CAS is send to service after restart the Broker, and how say
> >>>> >> parent CAS comes back but the child CAS doesnt. And if i try
> quiesce
> >>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> >>>> >> Remaining Number of CASes to Process:1". I suppose that is the
> >>>> >> child
> >>>> >> CAS.
> >>>> >>
> >>>> >> the client if can send more CASes to the service, but not receive
> >>>> >> response, for each CAS send after the one CAS that was send after
> >>>> >> restart broker (for which parent CAS comes back but the child CAS
> >>>> >> doesnt) not comes back parent CAS or child CAS.
> >>>> >>
> >>>> >> More specific:
> >>>> >> scenario 1:
> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>>> >>
> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show
> >>>> >> :
> >>>> >> "Jms Listener Failed." ....
> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >>>> >> comes back but the child CAS doesnt, to the uima-as client.)
> >>>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >>>> >>
> >>>> >> scenario 2:
> >>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>>> >>
> >>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show
> >>>> >> :
> >>>> >> "Jms Listener Failed." ....
> >>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >>>> >> comes back but the child CAS doesnt, to the uima-as client)
> >>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not
> comes
> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not
> comes
> >>>> >> back parent CAS or child CAS , to the uima-as client)
> >>>> >> If i try quiesce the Aggregate Service console shows: "Quiescing
> >>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >>>> >> > When a parent CAS comes back (after broker is restored), can the
> >>>> client
> >>>> >> > send more CASes to the service and receive responses? I am trying
> >>>> >> > to
> >>>> >> > determine what the real problem is here.
> >>>> >> >
> >>>> >> > 1) JMS Listener not recovering connection
> >>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
> >>>> connection is
> >>>> >> > stale
> >>>> >> >
> >>>> >> > -jerry
> >>>> >> >
> >>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
> >>>> uimaee@gmail.com>
> >>>> >> > wrote:
> >>>> >> >
> >>>> >> >> Strange. You say that the parent CAS comes back but the child
> CAS
> >>>> >> doesnt.
> >>>> >> >> Are there any exceptions in the service log? Perhaps the service
> >>>> >> >> is
> >>>> >> >> unable
> >>>> >> >> to send the child CAS due to missing broker. There should be
> >>>> something
> >>>> >> in
> >>>> >> >> the uima-as log about this.
> >>>> >> >>
> >>>> >> >> -jerry
> >>>> >> >>
> >>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >>>> >> >> <ne...@gmail.com>
> >>>> >> >> wrote:
> >>>> >> >>
> >>>> >> >>> Yes, it is supposed to do the service, given a
> >>>> >> >>> CAS it produces a child CAS and and this CAS continue in the
> >>>> >> >>> flow,
> >>>> >> >>> that are other primitive annotators. And I expected that the
> >>>> >> >>> child
> >>>> >> >>> CAS
> >>>> >> >>> will sent to UIMA-AS client, but not do.
> >>>> >> >>>
> >>>> >> >>> the scenario:
> >>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >>>> >> >>> everything is fine. The process is correct.
> >>>> >> >>>
> >>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log
> >>>> show :
> >>>> >> >>> "Jms Listener Failed." ....
> >>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >>>> >> >>>
> >>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >>>> >> >>>
> >>>> >> >>> Thank for you quick reply
> >>>> >> >>>
> >>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
> >:
> >>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
> >>>> Meaning
> >>>> >> >>> given a
> >>>> >> >>> > CAS it produces child CASes
> >>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
> >>>> service is
> >>>> >> >>> > supposed to do if the broker is running?
> >>>> >> >>> >
> >>>> >> >>> > Can you confirm the scenario:
> >>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >>>> >> >>> > 2) Service receives the CAS
> >>>> >> >>> > 3) Broker dies
> >>>> >> >>> > 4) Broker is restarted
> >>>> >> >>> >
> >>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when
> a
> >>>> >> >>> > broker
> >>>> >> >>> > is
> >>>> >> >>> > restarted?
> >>>> >> >>> >
> >>>> >> >>> > -Jerry
> >>>> >> >>> >
> >>>> >> >>> >
> >>>> >> >>> >
> >>>> >> >>> >
> >>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >>>> >> >>> nelsonrivera12@gmail.com>
> >>>> >> >>> > wrote:
> >>>> >> >>> >
> >>>> >> >>> >> I will try the variants that you say. I was deepening the
> >>>> problem:
> >>>> >> >>> >>
> >>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
> >>>> >> >>> >> original
> >>>> >> >>> >> CAS
> >>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
> >>>> generates
> >>>> >> >>> >> new
> >>>> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
> >>>> >> >>> >>
> >>>> >> >>> >> After restart broker (without restart the service of
> >>>> >> >>> >> annotator)
> >>>> >> >>> >> and
> >>>> >> >>> >> send a CAS to process, the listener associated to the
> client
> >>>> api
> >>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when
> >>>> the
> >>>> >> >>> >> original CAS finalizes, but the new Cas generated in the
> >>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >>>> >> >>> >>
> >>>> >> >>> >> with Aggregate AE simple the log is the same, but how the
> >>>> original
> >>>> >> >>> >> CAS
> >>>> >> >>> >> is notifies into the listener when finalizes, not have
> >>>> >> >>> >> problem
> >>>> >> >>> >> because is not expected for another new CAS.
> >>>> >> >>> >>
> >>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
> >>>> >> >>> >> <uimaee@gmail.com
> >>>> >:
> >>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use
> >>>> >> >>> >> > that
> >>>> >> >>> >> > broker
> >>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was
> >>>> >> >>> >> > a
> >>>> >> >>> >> > nasty
> >>>> >> >>> bug
> >>>> >> >>> >> > in
> >>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am not
> >>>> sure if
> >>>> >> >>> >> > this
> >>>> >> >>> >> > is
> >>>> >> >>> >> > the cause but worth trying.
> >>>> >> >>> >> >
> >>>> >> >>> >> > You can download AMQ from here:
> >>>> >> >>> >> > http://activemq.apache.org/download.html
> >>>> >> >>> >> >
> >>>> >> >>> >> > -Jerry
> >>>> >> >>> >> >
> >>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >>>> >> >>> >> > <ui...@gmail.com>
> >>>> >> >>> >> wrote:
> >>>> >> >>> >> >
> >>>> >> >>> >> >> Not sure what is causing this. I need to stage this with
> >>>> >> >>> >> >> UIMA-AS
> >>>> >> >>> >> >> 2.8.1.
> >>>> >> >>> >> I
> >>>> >> >>> >> >> am currently testing with trunk (latest) code. The most
> >>>> current
> >>>> >> >>> >> >> code
> >>>> >> >>> >> does
> >>>> >> >>> >> >> have newer broker.
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> I've started a broker and a sample service:
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> After the service started, I stopped the broker, waited a
> >>>> bit,
> >>>> >> and
> >>>> >> >>> >> >> restarted the broker.
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >>>> >> >>> >> >> Controller:
> >>>> >> >>> >> >> Meeting
> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >>>> >> >>> >> >> tcp://localhost:61616
> >>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
> >>>> >> >>> >> >> Controller:
> >>>> >> >>> >> >> Meeting
> >>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >>>> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> WARNING:
> >>>> >> Service:
> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> WARNING:
> >>>> Jms
> >>>> >> >>> Listener
> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >>>> >> >>> >> >> tcp://localhost:61616
> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >>>> ainer.handleListenerSetupFailure:
> >>>> >> >>> >> WARNING:
> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
> >>>> Connect
> >>>> >> To
> >>>> >> >>> >> >> Broker:
> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >>>> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> WARNING:
> >>>> >> Service:
> >>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
> WARNING:
> >>>> Jms
> >>>> >> >>> Listener
> >>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >>>> >> >>> >> >> tcp://localhost:61616
> >>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >>>> ainer.handleListenerSetupFailure:
> >>>> >> >>> >> WARNING:
> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
> >>>> Connect
> >>>> >> To
> >>>> >> >>> >> >> Broker:
> >>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >>>> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >>>> ainer.handleListenerSetupFailure:
> >>>> >> >>> >> WARNING:
> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> >>>> >> Connection
> >>>> >> >>> to
> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >>>> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >>>> >> >>> >> >> UimaDefaultMessageListenerCont
> >>>> ainer.handleListenerSetupFailure:
> >>>> >> >>> >> WARNING:
> >>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> >>>> >> Connection
> >>>> >> >>> to
> >>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >>>> >> >>> >> >>
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> I will repeat the above test with v 2.8.1
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> -Jerry
> >>>> >> >>> >> >>
> >>>> >> >>> >> >>
> >>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >>>> >> >>> >> nelsonrivera12@gmail.com>
> >>>> >> >>> >> >> wrote:
> >>>> >> >>> >> >>
> >>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >>>> >> >>> >> >>>
> >>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
> >>>> uimaee@gmail.com
> >>>> >> >:
> >>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >>>> >> >>> >> >>> >
> >>>> >> >>> >> >>> > -Jerry
> >>>> >> >>> >> >>> >
> >>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >>>> >> >>> >> >>> nelsonrivera12@gmail.com>
> >>>> >> >>> >> >>> > wrote:
> >>>> >> >>> >> >>> >
> >>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
> >>>> >> >>> >> >>> >> service,
> >>>> >> >>> >> >>> >> processing
> >>>> >> >>> >> >>> >> is
> >>>> >> >>> >> >>> >> executed correctly, but when the broker is stopped
> >>>> >> >>> >> >>> >> and
> >>>> >> >>> >> >>> >> subsequently
> >>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and
> after
> >>>> this
> >>>> >> >>> >> >>> >> the
> >>>> >> >>> >> >>> >> api
> >>>> >> >>> >> >>> >> client uima-as is not notified in the listener when a
> >>>> cas
> >>>> >> >>> >> >>> >> is
> >>>> >> >>> >> >>> >> processed, any more.
> >>>> >> >>> >> >>> >> can help me?:
> >>>> >> >>> >> >>> >>
> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >>>> >> >>> >> >>> >> Runtime
> >>>> >> >>> Exception
> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>>> >> >>> >> XClusterAnalyzerAggregate
> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>>> >> >>> javax.jms.JMSException:
> >>>> >> >>> >> >>> >> java.io.EOFException
> >>>> >> >>> >> >>> >> 02:08:25.570 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> >> >>> nerContainer.
> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >>>> >> >>> >> >>> >> Analyzer
> >>>> >> >>> >> >>> >> Listener
> >>>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
> >>>> Retrying
> >>>> >> >>> >> >>> >> ...
> >>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
> >>>> Endpoint:
> >>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> >>>> M1-33610-1474481197190-1:1:1
> >>>> >> >>> Managed
> >>>> >> >>> >> >>> >> By:
> >>>> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >>>> >> >>> >> >>> >> java.io.EOFException
> >>>> >> >>> >> >>> >> 02:09:02.222 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> >> >>> nerContainer.
> >>>> >> >>> >> >>> >> handleListenerSetupFailure:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
> >>>> >> >>> >> >>> >> Analyzer
> >>>> >> >>> >> >>> >> Listener
> >>>> >> >>> >> >>> >> Established Connection to Broker:
> >>>> >> >>> >> >>> >> tcp://localhost:61616
> >>>> >> >>> >> >>> >>
> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >>>> >> >>> >> >>> >> Runtime
> >>>> >> >>> Exception
> >>>> >> >>> >> >>> >> 02:09:02.317 - 14:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>>> >> >>> >> XClusterAnalyzerAggregate
> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
> >>>> >> connection
> >>>> >> >>> is
> >>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
> >>>> >> >>> >> >>> >> Runtime
> >>>> >> >>> Exception
> >>>> >> >>> >> >>> >> 02:09:02.357 - 15:
> >>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
> >>>> tivemq.UimaDefaultMessageListe
> >>>> >> >>> nerCont
> >>>> >> >>> >> >>> >> ainer.onException:
> >>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>>> >> >>> >> XClusterAnalyzerAggregate
> >>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
> >>>> closed.
> >>>> >> >>> >> >>> >>
> >>>> >> >>> >> >>> >
> >>>> >> >>> >> >>>
> >>>> >> >>> >> >>
> >>>> >> >>> >> >>
> >>>> >> >>> >> >
> >>>> >> >>> >>
> >>>> >> >>> >
> >>>> >> >>>
> >>>> >> >>
> >>>> >> >>
> >>>> >> >
> >>>> >>
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
My service is a aggregate analysis engine that receives input CAS
with the direcction of a directory of documents, the first component
is a MultiplerCas that for each document generate a child CAS, each
child CAS continues the flow that are other primitive annotators
(primitive annotators in c++ that can throw exceptions), then exists
other MultiplerCas that acts as Merger, waiting the complete
processing of each child CAS and finally generate a child CAS where
each child CAS previously processed is a view of CAS, this a child CAS
that is composed of view continúa el flujo a otro AE, que espera un
CAS de esta manera.
I will send you my uima-as descriptor, to understand better my
service. All delegates are co-located.
i will trys with uima-as-2.9.0 snapshot how you say, to see how it behaves
2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> I was able to stage this scenario in which an aggregate service cas
> multiplier receives a single input CAS and
> produces a child CAS.
>
> Scenario:
> 1) Service receives input CAS
> 2) CasMultiplier generates a child CAS
> 3) AE.process() is called (child CAS) and it sleeps for awhile
> 4) Broker is shutdown
> 5) AE.process() awakes from sleep
> 6) Uima-AS tries to deliver child CAS to a remote client
> 7) Connection is invalid so the child CAS is dropped
> 8) Broker is restarted
> 9) Listeners reconnect:
> INFO | Successfully refreshed JMS Connection
> INFO | Successfully refreshed JMS Connection
> 10) Broker log shows:
> WARN Service - Async error occurred:
> org.apache.activemq.DestinationDoesNotExistException: The destination
> temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> WARN Service - Async error occurred:
> org.apache.activemq.DestinationDoesNotExistException: The destination
> temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
>
> The service failed to send a reply.Nothing is delivered to a client as
> expected. This is the confusing part since you see a parent CAS being sent
> to a client. I just cant imagine how this is possible.
>
> When a service receives an input CAS in a message, it extracts a JMS
> replyTo object which is a JMS temp queue and stores in an internal cache
> until it is time to send a reply. Temp queues are transient and unique each
> time one is created.
>
> If the broker dies and subsequently restarts, the cached replyTo queue
> object is invalid (temp queue no longer exist) which leads to
> "Async error occurred:
> org.apache.activemq.DestinationDoesNotExistException" as shows above.
>
> In the service log I see a bunch of these exceptions:
>
> 02:05:46.574 - 56:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
> WARNING: Service: Test Aggregate TAE Unable to Open Connection To Broker:
> tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
> Retrying ...
> 02:05:46.574 - 56:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
> WARNING:
> javax.jms.JMSException: Could not connect to broker URL:
> tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
> java.net.ConnectException: Connection refused
>
> This is *after* the service is restarted so there seems to be a problem
> with reestablishing the connection to a new broker.
> I am still investigating that part. I just am confused how you are able to
> see a reply in the client.
>
> -Jerry
>
>
> On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
>
>> From what I understand there are two issues:
>>
>> 1) Why service JMS listeners are not reconnecting when a broker is
>> bounced. This
>> should be done by default. The 2.9.0 snapshot may address this since
>> a
>> listener
>> uses newer AMQ jars (5.14.0)
>>
>> 2) What happens to a child CAS when a Broker disappears and the service
>> wants
>> to deliver it to a client that is not reachable. I need to
>> investigate
>> this part. I think
>> such CAS is dropped but that should be reflected in a log and also
>> the
>> input CAS
>> should be marked as failed. Again something for me to investigate.
>>
>> I will try to set up a test to see if I can create the scenario.It will
>> take me a bit of time
>> to set this up but I will let you know what happens.
>>
>> -jerry
>>
>> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> wrote:
>>
>>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>>> https://repository.apache.org/content/repositories/snapshots
>>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>>> 23.195233-6.tar.gz
>>>
>>> Can you briefly describe your service? Does your service have any remote
>>> delegates or all parts of the pipeline are in the same process?
>>> Do you see and exceptions in the service log?
>>>
>>> -jerry
>>>
>>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera
>>> <ne...@gmail.com>
>>> wrote:
>>>
>>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>>>> appears to be broker
>>>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
>>>> work
>>>>
>>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>>> > Can you install AMQ broker 5.14.0 to see if this is related to broker
>>>> > issues.
>>>> >
>>>> > -jerry
>>>> >
>>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>>>> nelsonrivera12@gmail.com>
>>>> > wrote:
>>>> >
>>>> >> just like that, the parent CAS comes back but the child CAS doesnt.
>>>> >>
>>>> >> And the child CAS if processed, because the uima-as log shows the
>>>> >> lines of the other annotators primitive of the flow processing the
>>>> >> child CAS, but not comes back to UIMA-AS client.
>>>> >>
>>>> >> uima-as log only shows
>>>> >> "Jms Listener Failed. Endpoint:
>>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed
>>>> By:
>>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>>> >> java.io.EOFException" ........
>>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>>> >> XClusterAnalyzerAggregate
>>>> >> Managed By: tcp://localhost:61616 Reason:
>>>> >> javax.jms.IllegalStateException: The Consumer is
>>>> >> closed..............
>>>> >>
>>>> >> The CAS is send to service after restart the Broker, and how say
>>>> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
>>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>>>> >> Remaining Number of CASes to Process:1". I suppose that is the
>>>> >> child
>>>> >> CAS.
>>>> >>
>>>> >> the client if can send more CASes to the service, but not receive
>>>> >> response, for each CAS send after the one CAS that was send after
>>>> >> restart broker (for which parent CAS comes back but the child CAS
>>>> >> doesnt) not comes back parent CAS or child CAS.
>>>> >>
>>>> >> More specific:
>>>> >> scenario 1:
>>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>>> >>
>>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show
>>>> >> :
>>>> >> "Jms Listener Failed." ....
>>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>>>> >> comes back but the child CAS doesnt, to the uima-as client.)
>>>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
>>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>>>> >>
>>>> >> scenario 2:
>>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>>> >>
>>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show
>>>> >> :
>>>> >> "Jms Listener Failed." ....
>>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>>>> >> comes back but the child CAS doesnt, to the uima-as client)
>>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
>>>> >> back parent CAS or child CAS , to the uima-as client)
>>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
>>>> >> back parent CAS or child CAS , to the uima-as client)
>>>> >> If i try quiesce the Aggregate Service console shows: "Quiescing
>>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>>> >> > When a parent CAS comes back (after broker is restored), can the
>>>> client
>>>> >> > send more CASes to the service and receive responses? I am trying
>>>> >> > to
>>>> >> > determine what the real problem is here.
>>>> >> >
>>>> >> > 1) JMS Listener not recovering connection
>>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
>>>> connection is
>>>> >> > stale
>>>> >> >
>>>> >> > -jerry
>>>> >> >
>>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>>>> uimaee@gmail.com>
>>>> >> > wrote:
>>>> >> >
>>>> >> >> Strange. You say that the parent CAS comes back but the child CAS
>>>> >> doesnt.
>>>> >> >> Are there any exceptions in the service log? Perhaps the service
>>>> >> >> is
>>>> >> >> unable
>>>> >> >> to send the child CAS due to missing broker. There should be
>>>> something
>>>> >> in
>>>> >> >> the uima-as log about this.
>>>> >> >>
>>>> >> >> -jerry
>>>> >> >>
>>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>>>> >> >> <ne...@gmail.com>
>>>> >> >> wrote:
>>>> >> >>
>>>> >> >>> Yes, it is supposed to do the service, given a
>>>> >> >>> CAS it produces a child CAS and and this CAS continue in the
>>>> >> >>> flow,
>>>> >> >>> that are other primitive annotators. And I expected that the
>>>> >> >>> child
>>>> >> >>> CAS
>>>> >> >>> will sent to UIMA-AS client, but not do.
>>>> >> >>>
>>>> >> >>> the scenario:
>>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>>>> >> >>> everything is fine. The process is correct.
>>>> >> >>>
>>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log
>>>> show :
>>>> >> >>> "Jms Listener Failed." ....
>>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>>>> >> >>>
>>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>>>> >> >>>
>>>> >> >>> Thank for you quick reply
>>>> >> >>>
>>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
>>>> Meaning
>>>> >> >>> given a
>>>> >> >>> > CAS it produces child CASes
>>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
>>>> service is
>>>> >> >>> > supposed to do if the broker is running?
>>>> >> >>> >
>>>> >> >>> > Can you confirm the scenario:
>>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>>>> >> >>> > 2) Service receives the CAS
>>>> >> >>> > 3) Broker dies
>>>> >> >>> > 4) Broker is restarted
>>>> >> >>> >
>>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when a
>>>> >> >>> > broker
>>>> >> >>> > is
>>>> >> >>> > restarted?
>>>> >> >>> >
>>>> >> >>> > -Jerry
>>>> >> >>> >
>>>> >> >>> >
>>>> >> >>> >
>>>> >> >>> >
>>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>>>> >> >>> nelsonrivera12@gmail.com>
>>>> >> >>> > wrote:
>>>> >> >>> >
>>>> >> >>> >> I will try the variants that you say. I was deepening the
>>>> problem:
>>>> >> >>> >>
>>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
>>>> >> >>> >> original
>>>> >> >>> >> CAS
>>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
>>>> generates
>>>> >> >>> >> new
>>>> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
>>>> >> >>> >>
>>>> >> >>> >> After restart broker (without restart the service of
>>>> >> >>> >> annotator)
>>>> >> >>> >> and
>>>> >> >>> >> send a CAS to process, the listener associated to the client
>>>> api
>>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when
>>>> the
>>>> >> >>> >> original CAS finalizes, but the new Cas generated in the
>>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>>>> >> >>> >>
>>>> >> >>> >> with Aggregate AE simple the log is the same, but how the
>>>> original
>>>> >> >>> >> CAS
>>>> >> >>> >> is notifies into the listener when finalizes, not have
>>>> >> >>> >> problem
>>>> >> >>> >> because is not expected for another new CAS.
>>>> >> >>> >>
>>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik
>>>> >> >>> >> <uimaee@gmail.com
>>>> >:
>>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use
>>>> >> >>> >> > that
>>>> >> >>> >> > broker
>>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was
>>>> >> >>> >> > a
>>>> >> >>> >> > nasty
>>>> >> >>> bug
>>>> >> >>> >> > in
>>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am not
>>>> sure if
>>>> >> >>> >> > this
>>>> >> >>> >> > is
>>>> >> >>> >> > the cause but worth trying.
>>>> >> >>> >> >
>>>> >> >>> >> > You can download AMQ from here:
>>>> >> >>> >> > http://activemq.apache.org/download.html
>>>> >> >>> >> >
>>>> >> >>> >> > -Jerry
>>>> >> >>> >> >
>>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>>>> >> >>> >> > <ui...@gmail.com>
>>>> >> >>> >> wrote:
>>>> >> >>> >> >
>>>> >> >>> >> >> Not sure what is causing this. I need to stage this with
>>>> >> >>> >> >> UIMA-AS
>>>> >> >>> >> >> 2.8.1.
>>>> >> >>> >> I
>>>> >> >>> >> >> am currently testing with trunk (latest) code. The most
>>>> current
>>>> >> >>> >> >> code
>>>> >> >>> >> does
>>>> >> >>> >> >> have newer broker.
>>>> >> >>> >> >>
>>>> >> >>> >> >> I've started a broker and a sample service:
>>>> >> >>> >> >>
>>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>>>> >> >>> >> >>
>>>> >> >>> >> >> After the service started, I stopped the broker, waited a
>>>> bit,
>>>> >> and
>>>> >> >>> >> >> restarted the broker.
>>>> >> >>> >> >>
>>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
>>>> >> >>> >> >>
>>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>>>> >> >>> >> >> Controller:
>>>> >> >>> >> >> Meeting
>>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>>>> >> >>> >> >> tcp://localhost:61616
>>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
>>>> >> >>> >> >> Controller:
>>>> >> >>> >> >> Meeting
>>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>>>> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>>> >> Service:
>>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>>> Jms
>>>> >> >>> Listener
>>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>>> >> >>> >> >> tcp://localhost:61616
>>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>>> ainer.handleListenerSetupFailure:
>>>> >> >>> >> WARNING:
>>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>>>> Connect
>>>> >> To
>>>> >> >>> >> >> Broker:
>>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>>> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>>> >> Service:
>>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>>> Jms
>>>> >> >>> Listener
>>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>>> >> >>> >> >> tcp://localhost:61616
>>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>>> ainer.handleListenerSetupFailure:
>>>> >> >>> >> WARNING:
>>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>>>> Connect
>>>> >> To
>>>> >> >>> >> >> Broker:
>>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>>> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>>> ainer.handleListenerSetupFailure:
>>>> >> >>> >> WARNING:
>>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>>>> >> Connection
>>>> >> >>> to
>>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>>> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>>> ainer.handleListenerSetupFailure:
>>>> >> >>> >> WARNING:
>>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>>>> >> Connection
>>>> >> >>> to
>>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>>> >> >>> >> >>
>>>> >> >>> >> >>
>>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>>>> >> >>> >> >>
>>>> >> >>> >> >> -Jerry
>>>> >> >>> >> >>
>>>> >> >>> >> >>
>>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>>>> >> >>> >> nelsonrivera12@gmail.com>
>>>> >> >>> >> >> wrote:
>>>> >> >>> >> >>
>>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>>>> >> >>> >> >>>
>>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>>>> uimaee@gmail.com
>>>> >> >:
>>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>>>> >> >>> >> >>> >
>>>> >> >>> >> >>> > -Jerry
>>>> >> >>> >> >>> >
>>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>>>> >> >>> >> >>> > wrote:
>>>> >> >>> >> >>> >
>>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as
>>>> >> >>> >> >>> >> service,
>>>> >> >>> >> >>> >> processing
>>>> >> >>> >> >>> >> is
>>>> >> >>> >> >>> >> executed correctly, but when the broker is stopped
>>>> >> >>> >> >>> >> and
>>>> >> >>> >> >>> >> subsequently
>>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after
>>>> this
>>>> >> >>> >> >>> >> the
>>>> >> >>> >> >>> >> api
>>>> >> >>> >> >>> >> client uima-as is not notified in the listener when a
>>>> cas
>>>> >> >>> >> >>> >> is
>>>> >> >>> >> >>> >> processed, any more.
>>>> >> >>> >> >>> >> can help me?:
>>>> >> >>> >> >>> >>
>>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>>>> >> >>> >> >>> >> Runtime
>>>> >> >>> Exception
>>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>>> >> >>> >> XClusterAnalyzerAggregate
>>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>>> >> >>> javax.jms.JMSException:
>>>> >> >>> >> >>> >> java.io.EOFException
>>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> >> >>> nerContainer.
>>>> >> >>> >> >>> >> handleListenerSetupFailure:
>>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>>>> >> >>> >> >>> >> Analyzer
>>>> >> >>> >> >>> >> Listener
>>>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
>>>> Retrying
>>>> >> >>> >> >>> >> ...
>>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
>>>> Endpoint:
>>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>>>> M1-33610-1474481197190-1:1:1
>>>> >> >>> Managed
>>>> >> >>> >> >>> >> By:
>>>> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>>> >> >>> >> >>> >> java.io.EOFException
>>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> >> >>> nerContainer.
>>>> >> >>> >> >>> >> handleListenerSetupFailure:
>>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster
>>>> >> >>> >> >>> >> Analyzer
>>>> >> >>> >> >>> >> Listener
>>>> >> >>> >> >>> >> Established Connection to Broker:
>>>> >> >>> >> >>> >> tcp://localhost:61616
>>>> >> >>> >> >>> >>
>>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>>>> >> >>> >> >>> >> Runtime
>>>> >> >>> Exception
>>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>>> >> >>> >> XClusterAnalyzerAggregate
>>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
>>>> >> connection
>>>> >> >>> is
>>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer
>>>> >> >>> >> >>> >> Runtime
>>>> >> >>> Exception
>>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>>> tivemq.UimaDefaultMessageListe
>>>> >> >>> nerCont
>>>> >> >>> >> >>> >> ainer.onException:
>>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>>> >> >>> >> XClusterAnalyzerAggregate
>>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
>>>> closed.
>>>> >> >>> >> >>> >>
>>>> >> >>> >> >>> >
>>>> >> >>> >> >>>
>>>> >> >>> >> >>
>>>> >> >>> >> >>
>>>> >> >>> >> >
>>>> >> >>> >>
>>>> >> >>> >
>>>> >> >>>
>>>> >> >>
>>>> >> >>
>>>> >> >
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
I was able to stage this scenario in which an aggregate service cas
multiplier receives a single input CAS and
produces a child CAS.
Scenario:
1) Service receives input CAS
2) CasMultiplier generates a child CAS
3) AE.process() is called (child CAS) and it sleeps for awhile
4) Broker is shutdown
5) AE.process() awakes from sleep
6) Uima-AS tries to deliver child CAS to a remote client
7) Connection is invalid so the child CAS is dropped
8) Broker is restarted
9) Listeners reconnect:
INFO | Successfully refreshed JMS Connection
INFO | Successfully refreshed JMS Connection
10) Broker log shows:
WARN Service - Async error occurred:
org.apache.activemq.DestinationDoesNotExistException: The destination
temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
WARN Service - Async error occurred:
org.apache.activemq.DestinationDoesNotExistException: The destination
temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
The service failed to send a reply.Nothing is delivered to a client as
expected. This is the confusing part since you see a parent CAS being sent
to a client. I just cant imagine how this is possible.
When a service receives an input CAS in a message, it extracts a JMS
replyTo object which is a JMS temp queue and stores in an internal cache
until it is time to send a reply. Temp queues are transient and unique each
time one is created.
If the broker dies and subsequently restarts, the cached replyTo queue
object is invalid (temp queue no longer exist) which leads to
"Async error occurred:
org.apache.activemq.DestinationDoesNotExistException" as shows above.
In the service log I see a bunch of these exceptions:
02:05:46.574 - 56:
org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
WARNING: Service: Test Aggregate TAE Unable to Open Connection To Broker:
tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
Retrying ...
02:05:46.574 - 56:
org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
WARNING:
javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
java.net.ConnectException: Connection refused
This is *after* the service is restarted so there seems to be a problem
with reestablishing the connection to a new broker.
I am still investigating that part. I just am confused how you are able to
see a reply in the client.
-Jerry
On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
> From what I understand there are two issues:
>
> 1) Why service JMS listeners are not reconnecting when a broker is
> bounced. This
> should be done by default. The 2.9.0 snapshot may address this since a
> listener
> uses newer AMQ jars (5.14.0)
>
> 2) What happens to a child CAS when a Broker disappears and the service
> wants
> to deliver it to a client that is not reachable. I need to investigate
> this part. I think
> such CAS is dropped but that should be reflected in a log and also the
> input CAS
> should be marked as failed. Again something for me to investigate.
>
> I will try to set up a test to see if I can create the scenario.It will
> take me a bit of time
> to set this up but I will let you know what happens.
>
> -jerry
>
> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> wrote:
>
>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> https://repository.apache.org/content/repositories/snapshots
>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> 23.195233-6.tar.gz
>>
>> Can you briefly describe your service? Does your service have any remote
>> delegates or all parts of the pipeline are in the same process?
>> Do you see and exceptions in the service log?
>>
>> -jerry
>>
>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <ne...@gmail.com>
>> wrote:
>>
>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>>> appears to be broker
>>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
>>> work
>>>
>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> > Can you install AMQ broker 5.14.0 to see if this is related to broker
>>> > issues.
>>> >
>>> > -jerry
>>> >
>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>>> nelsonrivera12@gmail.com>
>>> > wrote:
>>> >
>>> >> just like that, the parent CAS comes back but the child CAS doesnt.
>>> >>
>>> >> And the child CAS if processed, because the uima-as log shows the
>>> >> lines of the other annotators primitive of the flow processing the
>>> >> child CAS, but not comes back to UIMA-AS client.
>>> >>
>>> >> uima-as log only shows
>>> >> "Jms Listener Failed. Endpoint:
>>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed
>>> By:
>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>> >> java.io.EOFException" ........
>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> javax.jms.IllegalStateException: The Consumer is closed..............
>>> >>
>>> >> The CAS is send to service after restart the Broker, and how say
>>> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
>>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>>> >> Remaining Number of CASes to Process:1". I suppose that is the child
>>> >> CAS.
>>> >>
>>> >> the client if can send more CASes to the service, but not receive
>>> >> response, for each CAS send after the one CAS that was send after
>>> >> restart broker (for which parent CAS comes back but the child CAS
>>> >> doesnt) not comes back parent CAS or child CAS.
>>> >>
>>> >> More specific:
>>> >> scenario 1:
>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>> >>
>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
>>> >> "Jms Listener Failed." ....
>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>>> >> comes back but the child CAS doesnt, to the uima-as client.)
>>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
>>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>>> >>
>>> >> scenario 2:
>>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>> >>
>>> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
>>> >> "Jms Listener Failed." ....
>>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>>> >> comes back but the child CAS doesnt, to the uima-as client)
>>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
>>> >> back parent CAS or child CAS , to the uima-as client)
>>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
>>> >> back parent CAS or child CAS , to the uima-as client)
>>> >> If i try quiesce the Aggregate Service console shows: "Quiescing
>>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> >> > When a parent CAS comes back (after broker is restored), can the
>>> client
>>> >> > send more CASes to the service and receive responses? I am trying to
>>> >> > determine what the real problem is here.
>>> >> >
>>> >> > 1) JMS Listener not recovering connection
>>> >> > 2) UIMA-AS is not handling CAS reply failures when broker
>>> connection is
>>> >> > stale
>>> >> >
>>> >> > -jerry
>>> >> >
>>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <
>>> uimaee@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> >> Strange. You say that the parent CAS comes back but the child CAS
>>> >> doesnt.
>>> >> >> Are there any exceptions in the service log? Perhaps the service is
>>> >> >> unable
>>> >> >> to send the child CAS due to missing broker. There should be
>>> something
>>> >> in
>>> >> >> the uima-as log about this.
>>> >> >>
>>> >> >> -jerry
>>> >> >>
>>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>>> >> >> <ne...@gmail.com>
>>> >> >> wrote:
>>> >> >>
>>> >> >>> Yes, it is supposed to do the service, given a
>>> >> >>> CAS it produces a child CAS and and this CAS continue in the flow,
>>> >> >>> that are other primitive annotators. And I expected that the child
>>> >> >>> CAS
>>> >> >>> will sent to UIMA-AS client, but not do.
>>> >> >>>
>>> >> >>> the scenario:
>>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>>> >> >>> everything is fine. The process is correct.
>>> >> >>>
>>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log
>>> show :
>>> >> >>> "Jms Listener Failed." ....
>>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>>> >> >>>
>>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>>> >> >>>
>>> >> >>> Thank for you quick reply
>>> >> >>>
>>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
>>> Meaning
>>> >> >>> given a
>>> >> >>> > CAS it produces child CASes
>>> >> >>> > which are then sent to UIMA-AS client. Is this what this
>>> service is
>>> >> >>> > supposed to do if the broker is running?
>>> >> >>> >
>>> >> >>> > Can you confirm the scenario:
>>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>>> >> >>> > 2) Service receives the CAS
>>> >> >>> > 3) Broker dies
>>> >> >>> > 4) Broker is restarted
>>> >> >>> >
>>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when a
>>> >> >>> > broker
>>> >> >>> > is
>>> >> >>> > restarted?
>>> >> >>> >
>>> >> >>> > -Jerry
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>>> >> >>> nelsonrivera12@gmail.com>
>>> >> >>> > wrote:
>>> >> >>> >
>>> >> >>> >> I will try the variants that you say. I was deepening the
>>> problem:
>>> >> >>> >>
>>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the original
>>> >> >>> >> CAS
>>> >> >>> >> is finish when it arrives to this CasMultiplier, this
>>> generates
>>> >> >>> >> new
>>> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
>>> >> >>> >>
>>> >> >>> >> After restart broker (without restart the service of annotator)
>>> >> >>> >> and
>>> >> >>> >> send a CAS to process, the listener associated to the client
>>> api
>>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when
>>> the
>>> >> >>> >> original CAS finalizes, but the new Cas generated in the
>>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>>> >> >>> >>
>>> >> >>> >> with Aggregate AE simple the log is the same, but how the
>>> original
>>> >> >>> >> CAS
>>> >> >>> >> is notifies into the listener when finalizes, not have problem
>>> >> >>> >> because is not expected for another new CAS.
>>> >> >>> >>
>>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>>> >:
>>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
>>> >> >>> >> > broker
>>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a
>>> >> >>> >> > nasty
>>> >> >>> bug
>>> >> >>> >> > in
>>> >> >>> >> > AMQ related to keep-alive protocol not working. I am not
>>> sure if
>>> >> >>> >> > this
>>> >> >>> >> > is
>>> >> >>> >> > the cause but worth trying.
>>> >> >>> >> >
>>> >> >>> >> > You can download AMQ from here:
>>> >> >>> >> > http://activemq.apache.org/download.html
>>> >> >>> >> >
>>> >> >>> >> > -Jerry
>>> >> >>> >> >
>>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>>> >> >>> >> > <ui...@gmail.com>
>>> >> >>> >> wrote:
>>> >> >>> >> >
>>> >> >>> >> >> Not sure what is causing this. I need to stage this with
>>> >> >>> >> >> UIMA-AS
>>> >> >>> >> >> 2.8.1.
>>> >> >>> >> I
>>> >> >>> >> >> am currently testing with trunk (latest) code. The most
>>> current
>>> >> >>> >> >> code
>>> >> >>> >> does
>>> >> >>> >> >> have newer broker.
>>> >> >>> >> >>
>>> >> >>> >> >> I've started a broker and a sample service:
>>> >> >>> >> >>
>>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>>> >> >>> >> >>
>>> >> >>> >> >> After the service started, I stopped the broker, waited a
>>> bit,
>>> >> and
>>> >> >>> >> >> restarted the broker.
>>> >> >>> >> >>
>>> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
>>> >> >>> >> >>
>>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>>> >> >>> >> >> Meeting
>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>>> >> >>> >> >> tcp://localhost:61616
>>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>>> >> >>> >> >> Meeting
>>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>>> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>> >> Service:
>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>> Jms
>>> >> >>> Listener
>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>> >> >>> >> >> tcp://localhost:61616
>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>> ainer.handleListenerSetupFailure:
>>> >> >>> >> WARNING:
>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>>> Connect
>>> >> To
>>> >> >>> >> >> Broker:
>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>> >> Service:
>>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>>> Jms
>>> >> >>> Listener
>>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>> >> >>> >> >> tcp://localhost:61616
>>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>> ainer.handleListenerSetupFailure:
>>> >> >>> >> WARNING:
>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>>> Connect
>>> >> To
>>> >> >>> >> >> Broker:
>>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>> ainer.handleListenerSetupFailure:
>>> >> >>> >> WARNING:
>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>>> >> Connection
>>> >> >>> to
>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >>> >> >> UimaDefaultMessageListenerCont
>>> ainer.handleListenerSetupFailure:
>>> >> >>> >> WARNING:
>>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>>> >> Connection
>>> >> >>> to
>>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >> I will repeat the above test with v 2.8.1
>>> >> >>> >> >>
>>> >> >>> >> >> -Jerry
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>>> >> >>> >> nelsonrivera12@gmail.com>
>>> >> >>> >> >> wrote:
>>> >> >>> >> >>
>>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>>> >> >>> >> >>>
>>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>>> uimaee@gmail.com
>>> >> >:
>>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>>> >> >>> >> >>> >
>>> >> >>> >> >>> > -Jerry
>>> >> >>> >> >>> >
>>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>>> >> >>> >> >>> nelsonrivera12@gmail.com>
>>> >> >>> >> >>> > wrote:
>>> >> >>> >> >>> >
>>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
>>> >> >>> >> >>> >> processing
>>> >> >>> >> >>> >> is
>>> >> >>> >> >>> >> executed correctly, but when the broker is stopped and
>>> >> >>> >> >>> >> subsequently
>>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after
>>> this
>>> >> >>> >> >>> >> the
>>> >> >>> >> >>> >> api
>>> >> >>> >> >>> >> client uima-as is not notified in the listener when a
>>> cas
>>> >> >>> >> >>> >> is
>>> >> >>> >> >>> >> processed, any more.
>>> >> >>> >> >>> >> can help me?:
>>> >> >>> >> >>> >>
>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> >> >>> Exception
>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> >>> >> XClusterAnalyzerAggregate
>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> >>> javax.jms.JMSException:
>>> >> >>> >> >>> >> java.io.EOFException
>>> >> >>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> >> >>> nerContainer.
>>> >> >>> >> >>> >> handleListenerSetupFailure:
>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>>> >> >>> >> >>> >> Listener
>>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
>>> Retrying
>>> >> >>> >> >>> >> ...
>>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed.
>>> Endpoint:
>>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>>> M1-33610-1474481197190-1:1:1
>>> >> >>> Managed
>>> >> >>> >> >>> >> By:
>>> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>> >> >>> >> >>> >> java.io.EOFException
>>> >> >>> >> >>> >> 02:09:02.222 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> >> >>> nerContainer.
>>> >> >>> >> >>> >> handleListenerSetupFailure:
>>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>>> >> >>> >> >>> >> Listener
>>> >> >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
>>> >> >>> >> >>> >>
>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> >> >>> Exception
>>> >> >>> >> >>> >> 02:09:02.317 - 14:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> >>> >> XClusterAnalyzerAggregate
>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
>>> >> connection
>>> >> >>> is
>>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> >> >>> Exception
>>> >> >>> >> >>> >> 02:09:02.357 - 15:
>>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>>> tivemq.UimaDefaultMessageListe
>>> >> >>> nerCont
>>> >> >>> >> >>> >> ainer.onException:
>>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> >>> >> XClusterAnalyzerAggregate
>>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is
>>> closed.
>>> >> >>> >> >>> >>
>>> >> >>> >> >>> >
>>> >> >>> >> >>>
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >
>>> >> >>> >>
>>> >> >>> >
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >
>>> >>
>>> >
>>>
>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
From what I understand there are two issues:
1) Why service JMS listeners are not reconnecting when a broker is bounced.
This
should be done by default. The 2.9.0 snapshot may address this since a
listener
uses newer AMQ jars (5.14.0)
2) What happens to a child CAS when a Broker disappears and the service
wants
to deliver it to a client that is not reachable. I need to investigate
this part. I think
such CAS is dropped but that should be reflected in a log and also the
input CAS
should be marked as failed. Again something for me to investigate.
I will try to set up a test to see if I can create the scenario.It will
take me a bit of time
to set this up but I will let you know what happens.
-jerry
On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
> Here is the uima-as 2.9.0-SNAPSHOT to try:
> https://repository.apache.org/content/repositories/snapshots
> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-
> 20160923.195233-6.tar.gz
>
> Can you briefly describe your service? Does your service have any remote
> delegates or all parts of the pipeline are in the same process?
> Do you see and exceptions in the service log?
>
> -jerry
>
> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>> appears to be broker
>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
>> work
>>
>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Can you install AMQ broker 5.14.0 to see if this is related to broker
>> > issues.
>> >
>> > -jerry
>> >
>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> nelsonrivera12@gmail.com>
>> > wrote:
>> >
>> >> just like that, the parent CAS comes back but the child CAS doesnt.
>> >>
>> >> And the child CAS if processed, because the uima-as log shows the
>> >> lines of the other annotators primitive of the flow processing the
>> >> child CAS, but not comes back to UIMA-AS client.
>> >>
>> >> uima-as log only shows
>> >> "Jms Listener Failed. Endpoint:
>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> java.io.EOFException" ........
>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason:
>> >> javax.jms.IllegalStateException: The Consumer is closed..............
>> >>
>> >> The CAS is send to service after restart the Broker, and how say
>> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>> >> Remaining Number of CASes to Process:1". I suppose that is the child
>> >> CAS.
>> >>
>> >> the client if can send more CASes to the service, but not receive
>> >> response, for each CAS send after the one CAS that was send after
>> >> restart broker (for which parent CAS comes back but the child CAS
>> >> doesnt) not comes back parent CAS or child CAS.
>> >>
>> >> More specific:
>> >> scenario 1:
>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>> >>
>> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
>> >> "Jms Listener Failed." ....
>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> >> comes back but the child CAS doesnt, to the uima-as client.)
>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >>
>> >> scenario 2:
>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>> >>
>> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
>> >> "Jms Listener Failed." ....
>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> >> comes back but the child CAS doesnt, to the uima-as client)
>> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
>> >> back parent CAS or child CAS , to the uima-as client)
>> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
>> >> back parent CAS or child CAS , to the uima-as client)
>> >> If i try quiesce the Aggregate Service console shows: "Quiescing
>> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > When a parent CAS comes back (after broker is restored), can the
>> client
>> >> > send more CASes to the service and receive responses? I am trying to
>> >> > determine what the real problem is here.
>> >> >
>> >> > 1) JMS Listener not recovering connection
>> >> > 2) UIMA-AS is not handling CAS reply failures when broker connection
>> is
>> >> > stale
>> >> >
>> >> > -jerry
>> >> >
>> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <uimaee@gmail.com
>> >
>> >> > wrote:
>> >> >
>> >> >> Strange. You say that the parent CAS comes back but the child CAS
>> >> doesnt.
>> >> >> Are there any exceptions in the service log? Perhaps the service is
>> >> >> unable
>> >> >> to send the child CAS due to missing broker. There should be
>> something
>> >> in
>> >> >> the uima-as log about this.
>> >> >>
>> >> >> -jerry
>> >> >>
>> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> >> <ne...@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Yes, it is supposed to do the service, given a
>> >> >>> CAS it produces a child CAS and and this CAS continue in the flow,
>> >> >>> that are other primitive annotators. And I expected that the child
>> >> >>> CAS
>> >> >>> will sent to UIMA-AS client, but not do.
>> >> >>>
>> >> >>> the scenario:
>> >> >>> 1- Start Broker and UIMA-AS aggregate service
>> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>> >> >>> everything is fine. The process is correct.
>> >> >>>
>> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log
>> show :
>> >> >>> "Jms Listener Failed." ....
>> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >> >>>
>> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>> >> >>>
>> >> >>> Thank for you quick reply
>> >> >>>
>> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
>> Meaning
>> >> >>> given a
>> >> >>> > CAS it produces child CASes
>> >> >>> > which are then sent to UIMA-AS client. Is this what this service
>> is
>> >> >>> > supposed to do if the broker is running?
>> >> >>> >
>> >> >>> > Can you confirm the scenario:
>> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >> >>> > 2) Service receives the CAS
>> >> >>> > 3) Broker dies
>> >> >>> > 4) Broker is restarted
>> >> >>> >
>> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when a
>> >> >>> > broker
>> >> >>> > is
>> >> >>> > restarted?
>> >> >>> >
>> >> >>> > -Jerry
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >> >>> nelsonrivera12@gmail.com>
>> >> >>> > wrote:
>> >> >>> >
>> >> >>> >> I will try the variants that you say. I was deepening the
>> problem:
>> >> >>> >>
>> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the original
>> >> >>> >> CAS
>> >> >>> >> is finish when it arrives to this CasMultiplier, this generates
>> >> >>> >> new
>> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
>> >> >>> >>
>> >> >>> >> After restart broker (without restart the service of annotator)
>> >> >>> >> and
>> >> >>> >> send a CAS to process, the listener associated to the client
>> api
>> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
>> >> >>> >> original CAS finalizes, but the new Cas generated in the
>> >> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >> >>> >>
>> >> >>> >> with Aggregate AE simple the log is the same, but how the
>> original
>> >> >>> >> CAS
>> >> >>> >> is notifies into the listener when finalizes, not have problem
>> >> >>> >> because is not expected for another new CAS.
>> >> >>> >>
>> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>> >:
>> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
>> >> >>> >> > broker
>> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a
>> >> >>> >> > nasty
>> >> >>> bug
>> >> >>> >> > in
>> >> >>> >> > AMQ related to keep-alive protocol not working. I am not sure
>> if
>> >> >>> >> > this
>> >> >>> >> > is
>> >> >>> >> > the cause but worth trying.
>> >> >>> >> >
>> >> >>> >> > You can download AMQ from here:
>> >> >>> >> > http://activemq.apache.org/download.html
>> >> >>> >> >
>> >> >>> >> > -Jerry
>> >> >>> >> >
>> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>> >> >>> >> > <ui...@gmail.com>
>> >> >>> >> wrote:
>> >> >>> >> >
>> >> >>> >> >> Not sure what is causing this. I need to stage this with
>> >> >>> >> >> UIMA-AS
>> >> >>> >> >> 2.8.1.
>> >> >>> >> I
>> >> >>> >> >> am currently testing with trunk (latest) code. The most
>> current
>> >> >>> >> >> code
>> >> >>> >> does
>> >> >>> >> >> have newer broker.
>> >> >>> >> >>
>> >> >>> >> >> I've started a broker and a sample service:
>> >> >>> >> >>
>> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >>> >> >>
>> >> >>> >> >> After the service started, I stopped the broker, waited a
>> bit,
>> >> and
>> >> >>> >> >> restarted the broker.
>> >> >>> >> >>
>> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
>> >> >>> >> >>
>> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>> >> >>> >> >> Meeting
>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >> >>> >> >> tcp://localhost:61616
>> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>> >> >>> >> >> Meeting
>> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >>> >> >> queue://MeetingDetectorTaeQueue
>> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> >> Service:
>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> Jms
>> >> >>> Listener
>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> >>> >> >> tcp://localhost:61616
>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailu
>> re:
>> >> >>> >> WARNING:
>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>> Connect
>> >> To
>> >> >>> >> >> Broker:
>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> >> Service:
>> >> >>> >> >> Meeting Detector TAE Runtime Exception
>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> Jms
>> >> >>> Listener
>> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> >>> >> >> tcp://localhost:61616
>> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailu
>> re:
>> >> >>> >> WARNING:
>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
>> Connect
>> >> To
>> >> >>> >> >> Broker:
>> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailu
>> re:
>> >> >>> >> WARNING:
>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> >> Connection
>> >> >>> to
>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailu
>> re:
>> >> >>> >> WARNING:
>> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> >> Connection
>> >> >>> to
>> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >> >>> >> >>
>> >> >>> >> >>
>> >> >>> >> >> I will repeat the above test with v 2.8.1
>> >> >>> >> >>
>> >> >>> >> >> -Jerry
>> >> >>> >> >>
>> >> >>> >> >>
>> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >> >>> >> nelsonrivera12@gmail.com>
>> >> >>> >> >> wrote:
>> >> >>> >> >>
>> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >> >>> >> >>>
>> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
>> uimaee@gmail.com
>> >> >:
>> >> >>> >> >>> > Which version of UIMA-AS are you using?
>> >> >>> >> >>> >
>> >> >>> >> >>> > -Jerry
>> >> >>> >> >>> >
>> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >> >>> >> >>> nelsonrivera12@gmail.com>
>> >> >>> >> >>> > wrote:
>> >> >>> >> >>> >
>> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
>> >> >>> >> >>> >> processing
>> >> >>> >> >>> >> is
>> >> >>> >> >>> >> executed correctly, but when the broker is stopped and
>> >> >>> >> >>> >> subsequently
>> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after
>> this
>> >> >>> >> >>> >> the
>> >> >>> >> >>> >> api
>> >> >>> >> >>> >> client uima-as is not notified in the listener when a cas
>> >> >>> >> >>> >> is
>> >> >>> >> >>> >> processed, any more.
>> >> >>> >> >>> >> can help me?:
>> >> >>> >> >>> >>
>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >> >>> Exception
>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>> javax.jms.JMSException:
>> >> >>> >> >>> >> java.io.EOFException
>> >> >>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> >> >>> nerContainer.
>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>> >> >>> >> >>> >> Listener
>> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
>> Retrying
>> >> >>> >> >>> >> ...
>> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
>> M1-33610-1474481197190-1:1:1
>> >> >>> Managed
>> >> >>> >> >>> >> By:
>> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >>> >> >>> >> java.io.EOFException
>> >> >>> >> >>> >> 02:09:02.222 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> >> >>> nerContainer.
>> >> >>> >> >>> >> handleListenerSetupFailure:
>> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>> >> >>> >> >>> >> Listener
>> >> >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
>> >> >>> >> >>> >>
>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >> >>> Exception
>> >> >>> >> >>> >> 02:09:02.317 - 14:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
>> >> connection
>> >> >>> is
>> >> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >> >>> Exception
>> >> >>> >> >>> >> 02:09:02.357 - 15:
>> >> >>> >> >>> >> org.apache.uima.adapter.jms.ac
>> tivemq.UimaDefaultMessageListe
>> >> >>> nerCont
>> >> >>> >> >>> >> ainer.onException:
>> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>> >> XClusterAnalyzerAggregate
>> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
>> >> >>> >> >>> >>
>> >> >>> >> >>> >
>> >> >>> >> >>>
>> >> >>> >> >>
>> >> >>> >> >>
>> >> >>> >> >
>> >> >>> >>
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >
>>
>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Here is the uima-as 2.9.0-SNAPSHOT to try:
https://repository.apache.org/content/repositories/
snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-
as-2.9.0-20160923.195233-6.tar.gz
Can you briefly describe your service? Does your service have any remote
delegates or all parts of the pipeline are in the same process?
Do you see and exceptions in the service log?
-jerry
On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <ne...@gmail.com>
wrote:
> I was testing with AMQ broker 5.14.0 and i get the same error. not
> appears to be broker
> issues, appears to be uima-as issues. Any other idea, i am stagnant at work
>
> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Can you install AMQ broker 5.14.0 to see if this is related to broker
> > issues.
> >
> > -jerry
> >
> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> nelsonrivera12@gmail.com>
> > wrote:
> >
> >> just like that, the parent CAS comes back but the child CAS doesnt.
> >>
> >> And the child CAS if processed, because the uima-as log shows the
> >> lines of the other annotators primitive of the flow processing the
> >> child CAS, but not comes back to UIMA-AS client.
> >>
> >> uima-as log only shows
> >> "Jms Listener Failed. Endpoint:
> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException" ........
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason:
> >> javax.jms.IllegalStateException: The Consumer is closed..............
> >>
> >> The CAS is send to service after restart the Broker, and how say
> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> >> Remaining Number of CASes to Process:1". I suppose that is the child
> >> CAS.
> >>
> >> the client if can send more CASes to the service, but not receive
> >> response, for each CAS send after the one CAS that was send after
> >> restart broker (for which parent CAS comes back but the child CAS
> >> doesnt) not comes back parent CAS or child CAS.
> >>
> >> More specific:
> >> scenario 1:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
> >> "Jms Listener Failed." ....
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client.)
> >> If i try quiesce the Aggregate Service console shows : "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >>
> >> scenario 2:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service). UIMA-AS log show :
> >> "Jms Listener Failed." ....
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client)
> >> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >> If i try quiesce the Aggregate Service console shows: "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >>
> >>
> >>
> >>
> >>
> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > When a parent CAS comes back (after broker is restored), can the
> client
> >> > send more CASes to the service and receive responses? I am trying to
> >> > determine what the real problem is here.
> >> >
> >> > 1) JMS Listener not recovering connection
> >> > 2) UIMA-AS is not handling CAS reply failures when broker connection
> is
> >> > stale
> >> >
> >> > -jerry
> >> >
> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <ui...@gmail.com>
> >> > wrote:
> >> >
> >> >> Strange. You say that the parent CAS comes back but the child CAS
> >> doesnt.
> >> >> Are there any exceptions in the service log? Perhaps the service is
> >> >> unable
> >> >> to send the child CAS due to missing broker. There should be
> something
> >> in
> >> >> the uima-as log about this.
> >> >>
> >> >> -jerry
> >> >>
> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >> <ne...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Yes, it is supposed to do the service, given a
> >> >>> CAS it produces a child CAS and and this CAS continue in the flow,
> >> >>> that are other primitive annotators. And I expected that the child
> >> >>> CAS
> >> >>> will sent to UIMA-AS client, but not do.
> >> >>>
> >> >>> the scenario:
> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >> >>> everything is fine. The process is correct.
> >> >>>
> >> >>> 3- Restart Broker (without restart the service). UIMA-AS log show
> :
> >> >>> "Jms Listener Failed." ....
> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >>>
> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >> >>>
> >> >>> Thank for you quick reply
> >> >>>
> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
> Meaning
> >> >>> given a
> >> >>> > CAS it produces child CASes
> >> >>> > which are then sent to UIMA-AS client. Is this what this service
> is
> >> >>> > supposed to do if the broker is running?
> >> >>> >
> >> >>> > Can you confirm the scenario:
> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >> >>> > 2) Service receives the CAS
> >> >>> > 3) Broker dies
> >> >>> > 4) Broker is restarted
> >> >>> >
> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when a
> >> >>> > broker
> >> >>> > is
> >> >>> > restarted?
> >> >>> >
> >> >>> > -Jerry
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >>> nelsonrivera12@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> >> I will try the variants that you say. I was deepening the
> problem:
> >> >>> >>
> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the original
> >> >>> >> CAS
> >> >>> >> is finish when it arrives to this CasMultiplier, this generates
> >> >>> >> new
> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
> >> >>> >>
> >> >>> >> After restart broker (without restart the service of annotator)
> >> >>> >> and
> >> >>> >> send a CAS to process, the listener associated to the client api
> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
> >> >>> >> original CAS finalizes, but the new Cas generated in the
> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >> >>> >>
> >> >>> >> with Aggregate AE simple the log is the same, but how the
> original
> >> >>> >> CAS
> >> >>> >> is notifies into the listener when finalizes, not have problem
> >> >>> >> because is not expected for another new CAS.
> >> >>> >>
> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
> >> >>> >> > broker
> >> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a
> >> >>> >> > nasty
> >> >>> bug
> >> >>> >> > in
> >> >>> >> > AMQ related to keep-alive protocol not working. I am not sure
> if
> >> >>> >> > this
> >> >>> >> > is
> >> >>> >> > the cause but worth trying.
> >> >>> >> >
> >> >>> >> > You can download AMQ from here:
> >> >>> >> > http://activemq.apache.org/download.html
> >> >>> >> >
> >> >>> >> > -Jerry
> >> >>> >> >
> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >> >>> >> > <ui...@gmail.com>
> >> >>> >> wrote:
> >> >>> >> >
> >> >>> >> >> Not sure what is causing this. I need to stage this with
> >> >>> >> >> UIMA-AS
> >> >>> >> >> 2.8.1.
> >> >>> >> I
> >> >>> >> >> am currently testing with trunk (latest) code. The most
> current
> >> >>> >> >> code
> >> >>> >> does
> >> >>> >> >> have newer broker.
> >> >>> >> >>
> >> >>> >> >> I've started a broker and a sample service:
> >> >>> >> >>
> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >>> >> >>
> >> >>> >> >> After the service started, I stopped the broker, waited a bit,
> >> and
> >> >>> >> >> restarted the broker.
> >> >>> >> >>
> >> >>> >> >> With UIMA-AS from trunk I see the following in the log:
> >> >>> >> >>
> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
> >> >>> >> >> Meeting
> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
> >> >>> >> >> Meeting
> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
> >> Service:
> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> >> >>> Listener
> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
> Connect
> >> To
> >> >>> >> >> Broker:
> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
> >> Service:
> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> >> >>> Listener
> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To
> Connect
> >> To
> >> >>> >> >> Broker:
> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> >> Connection
> >> >>> to
> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> >> Connection
> >> >>> to
> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >>> >> >>
> >> >>> >> >> -Jerry
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >> >>> >> nelsonrivera12@gmail.com>
> >> >>> >> >> wrote:
> >> >>> >> >>
> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >> >>> >> >>>
> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <
> uimaee@gmail.com
> >> >:
> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >>> >> >>> >
> >> >>> >> >>> > -Jerry
> >> >>> >> >>> >
> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >>> >> >>> > wrote:
> >> >>> >> >>> >
> >> >>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
> >> >>> >> >>> >> processing
> >> >>> >> >>> >> is
> >> >>> >> >>> >> executed correctly, but when the broker is stopped and
> >> >>> >> >>> >> subsequently
> >> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after
> this
> >> >>> >> >>> >> the
> >> >>> >> >>> >> api
> >> >>> >> >>> >> client uima-as is not notified in the listener when a cas
> >> >>> >> >>> >> is
> >> >>> >> >>> >> processed, any more.
> >> >>> >> >>> >> can help me?:
> >> >>> >> >>> >>
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>> javax.jms.JMSException:
> >> >>> >> >>> >> java.io.EOFException
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> >> >>> nerContainer.
> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
> >> >>> >> >>> >> Listener
> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
> Retrying
> >> >>> >> >>> >> ...
> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> M1-33610-1474481197190-1:1:1
> >> >>> Managed
> >> >>> >> >>> >> By:
> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >>> >> >>> >> java.io.EOFException
> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> >> >>> nerContainer.
> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
> >> >>> >> >>> >> Listener
> >> >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
> >> >>> >> >>> >>
> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
> >> connection
> >> >>> is
> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
> >> >>> >> >>> >>
> >> >>> >> >>> >
> >> >>> >> >>>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
I was testing with AMQ broker 5.14.0 and i get the same error. not
appears to be broker
issues, appears to be uima-as issues. Any other idea, i am stagnant at work
2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Can you install AMQ broker 5.14.0 to see if this is related to broker
> issues.
>
> -jerry
>
> On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> just like that, the parent CAS comes back but the child CAS doesnt.
>>
>> And the child CAS if processed, because the uima-as log shows the
>> lines of the other annotators primitive of the flow processing the
>> child CAS, but not comes back to UIMA-AS client.
>>
>> uima-as log only shows
>> "Jms Listener Failed. Endpoint:
>> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
>> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException" ........
>> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.IllegalStateException: The Consumer is closed..............
>>
>> The CAS is send to service after restart the Broker, and how say
>> parent CAS comes back but the child CAS doesnt. And if i try quiesce
>> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>> Remaining Number of CASes to Process:1". I suppose that is the child
>> CAS.
>>
>> the client if can send more CASes to the service, but not receive
>> response, for each CAS send after the one CAS that was send after
>> restart broker (for which parent CAS comes back but the child CAS
>> doesnt) not comes back parent CAS or child CAS.
>>
>> More specific:
>> scenario 1:
>> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>
>> 2- Restart Broker (without restart the service). UIMA-AS log show :
>> "Jms Listener Failed." ....
>> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> comes back but the child CAS doesnt, to the uima-as client.)
>> If i try quiesce the Aggregate Service console shows : "Quiescing
>> UIMA-AS Service. Remaining Number of CASes to Process:1"
>>
>> scenario 2:
>> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>>
>> 2- Restart Broker (without restart the service). UIMA-AS log show :
>> "Jms Listener Failed." ....
>> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> comes back but the child CAS doesnt, to the uima-as client)
>> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
>> back parent CAS or child CAS , to the uima-as client)
>> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
>> back parent CAS or child CAS , to the uima-as client)
>> If i try quiesce the Aggregate Service console shows: "Quiescing
>> UIMA-AS Service. Remaining Number of CASes to Process:5"
>>
>>
>>
>>
>>
>> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > When a parent CAS comes back (after broker is restored), can the client
>> > send more CASes to the service and receive responses? I am trying to
>> > determine what the real problem is here.
>> >
>> > 1) JMS Listener not recovering connection
>> > 2) UIMA-AS is not handling CAS reply failures when broker connection is
>> > stale
>> >
>> > -jerry
>> >
>> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <ui...@gmail.com>
>> > wrote:
>> >
>> >> Strange. You say that the parent CAS comes back but the child CAS
>> doesnt.
>> >> Are there any exceptions in the service log? Perhaps the service is
>> >> unable
>> >> to send the child CAS due to missing broker. There should be something
>> in
>> >> the uima-as log about this.
>> >>
>> >> -jerry
>> >>
>> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> >> <ne...@gmail.com>
>> >> wrote:
>> >>
>> >>> Yes, it is supposed to do the service, given a
>> >>> CAS it produces a child CAS and and this CAS continue in the flow,
>> >>> that are other primitive annotators. And I expected that the child
>> >>> CAS
>> >>> will sent to UIMA-AS client, but not do.
>> >>>
>> >>> the scenario:
>> >>> 1- Start Broker and UIMA-AS aggregate service
>> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>> >>> everything is fine. The process is correct.
>> >>>
>> >>> 3- Restart Broker (without restart the service). UIMA-AS log show :
>> >>> "Jms Listener Failed." ....
>> >>> 4- Client sends a CAS to UIMA-AS aggregate service
>> >>>
>> >>> UIMA-AS client receive the parent CAS, but not child CAS.
>> >>>
>> >>> Thank for you quick reply
>> >>>
>> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
>> >>> given a
>> >>> > CAS it produces child CASes
>> >>> > which are then sent to UIMA-AS client. Is this what this service is
>> >>> > supposed to do if the broker is running?
>> >>> >
>> >>> > Can you confirm the scenario:
>> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> >>> > 2) Service receives the CAS
>> >>> > 3) Broker dies
>> >>> > 4) Broker is restarted
>> >>> >
>> >>> > Does the UIMA-AS client receive the CAS back in a reply when a
>> >>> > broker
>> >>> > is
>> >>> > restarted?
>> >>> >
>> >>> > -Jerry
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> >>> nelsonrivera12@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> >> I will try the variants that you say. I was deepening the problem:
>> >>> >>
>> >>> >> I have a Aggregate AE that contains CasMultiplier, the original
>> >>> >> CAS
>> >>> >> is finish when it arrives to this CasMultiplier, this generates
>> >>> >> new
>> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
>> >>> >>
>> >>> >> After restart broker (without restart the service of annotator)
>> >>> >> and
>> >>> >> send a CAS to process, the listener associated to the client api
>> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
>> >>> >> original CAS finalizes, but the new Cas generated in the
>> >>> >> CasMultiplier, It is not notifies when finalizes.
>> >>> >>
>> >>> >> with Aggregate AE simple the log is the same, but how the original
>> >>> >> CAS
>> >>> >> is notifies into the listener when finalizes, not have problem
>> >>> >> because is not expected for another new CAS.
>> >>> >>
>> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
>> >>> >> > broker
>> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a
>> >>> >> > nasty
>> >>> bug
>> >>> >> > in
>> >>> >> > AMQ related to keep-alive protocol not working. I am not sure if
>> >>> >> > this
>> >>> >> > is
>> >>> >> > the cause but worth trying.
>> >>> >> >
>> >>> >> > You can download AMQ from here:
>> >>> >> > http://activemq.apache.org/download.html
>> >>> >> >
>> >>> >> > -Jerry
>> >>> >> >
>> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>> >>> >> > <ui...@gmail.com>
>> >>> >> wrote:
>> >>> >> >
>> >>> >> >> Not sure what is causing this. I need to stage this with
>> >>> >> >> UIMA-AS
>> >>> >> >> 2.8.1.
>> >>> >> I
>> >>> >> >> am currently testing with trunk (latest) code. The most current
>> >>> >> >> code
>> >>> >> does
>> >>> >> >> have newer broker.
>> >>> >> >>
>> >>> >> >> I've started a broker and a sample service:
>> >>> >> >>
>> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>> >>> >> >> $UIMA_HOME/examples/deploy/as/
>> >>> >> >> Deploy_MeetingDetectorTAE.xml
>> >>> >> >>
>> >>> >> >> After the service started, I stopped the broker, waited a bit,
>> and
>> >>> >> >> restarted the broker.
>> >>> >> >>
>> >>> >> >> With UIMA-AS from trunk I see the following in the log:
>> >>> >> >>
>> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>> >>> >> >> Meeting
>> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >>> >> >> queue://MeetingDetectorTaeQueue
>> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>> >>> >> >> tcp://localhost:61616
>> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>> >>> >> >> Meeting
>> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >>> >> >> queue://MeetingDetectorTaeQueue
>> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> Service:
>> >>> >> >> Meeting Detector TAE Runtime Exception
>> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>> >>> Listener
>> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >>> >> >> tcp://localhost:61616
>> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >>> >> WARNING:
>> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect
>> To
>> >>> >> >> Broker:
>> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
>> Service:
>> >>> >> >> Meeting Detector TAE Runtime Exception
>> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>> >>> Listener
>> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >>> >> >> tcp://localhost:61616
>> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >>> >> WARNING:
>> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect
>> To
>> >>> >> >> Broker:
>> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >>> >> WARNING:
>> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> Connection
>> >>> to
>> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >>> >> WARNING:
>> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
>> Connection
>> >>> to
>> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> I will repeat the above test with v 2.8.1
>> >>> >> >>
>> >>> >> >> -Jerry
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >>> >> nelsonrivera12@gmail.com>
>> >>> >> >> wrote:
>> >>> >> >>
>> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >>> >> >>>
>> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
>> >:
>> >>> >> >>> > Which version of UIMA-AS are you using?
>> >>> >> >>> >
>> >>> >> >>> > -Jerry
>> >>> >> >>> >
>> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >>> >> >>> nelsonrivera12@gmail.com>
>> >>> >> >>> > wrote:
>> >>> >> >>> >
>> >>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
>> >>> >> >>> >> processing
>> >>> >> >>> >> is
>> >>> >> >>> >> executed correctly, but when the broker is stopped and
>> >>> >> >>> >> subsequently
>> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after this
>> >>> >> >>> >> the
>> >>> >> >>> >> api
>> >>> >> >>> >> client uima-as is not notified in the listener when a cas
>> >>> >> >>> >> is
>> >>> >> >>> >> processed, any more.
>> >>> >> >>> >> can help me?:
>> >>> >> >>> >>
>> >>> >> >>> >> 02:08:25.570 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >>> Exception
>> >>> >> >>> >> 02:08:25.570 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>> >> XClusterAnalyzerAggregate
>> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>> javax.jms.JMSException:
>> >>> >> >>> >> java.io.EOFException
>> >>> >> >>> >> 02:08:25.570 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> >> >>> nerContainer.
>> >>> >> >>> >> handleListenerSetupFailure:
>> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>> >>> >> >>> >> Listener
>> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying
>> >>> >> >>> >> ...
>> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1
>> >>> Managed
>> >>> >> >>> >> By:
>> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >>> >> >>> >> java.io.EOFException
>> >>> >> >>> >> 02:09:02.222 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> >> >>> nerContainer.
>> >>> >> >>> >> handleListenerSetupFailure:
>> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>> >>> >> >>> >> Listener
>> >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
>> >>> >> >>> >>
>> >>> >> >>> >> 02:09:02.317 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >>> Exception
>> >>> >> >>> >> 02:09:02.317 - 14:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>> >> XClusterAnalyzerAggregate
>> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
>> connection
>> >>> is
>> >>> >> >>> >> already closed02:09:02.356 - 15:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> >>> Exception
>> >>> >> >>> >> 02:09:02.357 - 15:
>> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerCont
>> >>> >> >>> >> ainer.onException:
>> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>> >> XClusterAnalyzerAggregate
>> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
>> >>> >> >>> >>
>> >>> >> >>> >
>> >>> >> >>>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Can you install AMQ broker 5.14.0 to see if this is related to broker
issues.
-jerry
On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <ne...@gmail.com>
wrote:
> just like that, the parent CAS comes back but the child CAS doesnt.
>
> And the child CAS if processed, because the uima-as log shows the
> lines of the other annotators primitive of the flow processing the
> child CAS, but not comes back to UIMA-AS client.
>
> uima-as log only shows
> "Jms Listener Failed. Endpoint:
> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException" ........
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed..............
>
> The CAS is send to service after restart the Broker, and how say
> parent CAS comes back but the child CAS doesnt. And if i try quiesce
> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> Remaining Number of CASes to Process:1". I suppose that is the child
> CAS.
>
> the client if can send more CASes to the service, but not receive
> response, for each CAS send after the one CAS that was send after
> restart broker (for which parent CAS comes back but the child CAS
> doesnt) not comes back parent CAS or child CAS.
>
> More specific:
> scenario 1:
> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>
> 2- Restart Broker (without restart the service). UIMA-AS log show :
> "Jms Listener Failed." ....
> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> comes back but the child CAS doesnt, to the uima-as client.)
> If i try quiesce the Aggregate Service console shows : "Quiescing
> UIMA-AS Service. Remaining Number of CASes to Process:1"
>
> scenario 2:
> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>
> 2- Restart Broker (without restart the service). UIMA-AS log show :
> "Jms Listener Failed." ....
> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> comes back but the child CAS doesnt, to the uima-as client)
> 4- Client sends another CAS to UIMA-AS aggregate service.(not comes
> back parent CAS or child CAS , to the uima-as client)
> 5- Client sends another CAS to UIMA-AS aggregate service.(not comes
> back parent CAS or child CAS , to the uima-as client)
> If i try quiesce the Aggregate Service console shows: "Quiescing
> UIMA-AS Service. Remaining Number of CASes to Process:5"
>
>
>
>
>
> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > When a parent CAS comes back (after broker is restored), can the client
> > send more CASes to the service and receive responses? I am trying to
> > determine what the real problem is here.
> >
> > 1) JMS Listener not recovering connection
> > 2) UIMA-AS is not handling CAS reply failures when broker connection is
> > stale
> >
> > -jerry
> >
> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <ui...@gmail.com>
> > wrote:
> >
> >> Strange. You say that the parent CAS comes back but the child CAS
> doesnt.
> >> Are there any exceptions in the service log? Perhaps the service is
> >> unable
> >> to send the child CAS due to missing broker. There should be something
> in
> >> the uima-as log about this.
> >>
> >> -jerry
> >>
> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> <ne...@gmail.com>
> >> wrote:
> >>
> >>> Yes, it is supposed to do the service, given a
> >>> CAS it produces a child CAS and and this CAS continue in the flow,
> >>> that are other primitive annotators. And I expected that the child CAS
> >>> will sent to UIMA-AS client, but not do.
> >>>
> >>> the scenario:
> >>> 1- Start Broker and UIMA-AS aggregate service
> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >>> everything is fine. The process is correct.
> >>>
> >>> 3- Restart Broker (without restart the service). UIMA-AS log show :
> >>> "Jms Listener Failed." ....
> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >>>
> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >>>
> >>> Thank for you quick reply
> >>>
> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
> >>> given a
> >>> > CAS it produces child CASes
> >>> > which are then sent to UIMA-AS client. Is this what this service is
> >>> > supposed to do if the broker is running?
> >>> >
> >>> > Can you confirm the scenario:
> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >>> > 2) Service receives the CAS
> >>> > 3) Broker dies
> >>> > 4) Broker is restarted
> >>> >
> >>> > Does the UIMA-AS client receive the CAS back in a reply when a broker
> >>> > is
> >>> > restarted?
> >>> >
> >>> > -Jerry
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >>> nelsonrivera12@gmail.com>
> >>> > wrote:
> >>> >
> >>> >> I will try the variants that you say. I was deepening the problem:
> >>> >>
> >>> >> I have a Aggregate AE that contains CasMultiplier, the original CAS
> >>> >> is finish when it arrives to this CasMultiplier, this generates new
> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
> >>> >>
> >>> >> After restart broker (without restart the service of annotator) and
> >>> >> send a CAS to process, the listener associated to the client api
> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
> >>> >> original CAS finalizes, but the new Cas generated in the
> >>> >> CasMultiplier, It is not notifies when finalizes.
> >>> >>
> >>> >> with Aggregate AE simple the log is the same, but how the original
> >>> >> CAS
> >>> >> is notifies into the listener when finalizes, not have problem
> >>> >> because is not expected for another new CAS.
> >>> >>
> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
> >>> >> > broker
> >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty
> >>> bug
> >>> >> > in
> >>> >> > AMQ related to keep-alive protocol not working. I am not sure if
> >>> >> > this
> >>> >> > is
> >>> >> > the cause but worth trying.
> >>> >> >
> >>> >> > You can download AMQ from here:
> >>> >> > http://activemq.apache.org/download.html
> >>> >> >
> >>> >> > -Jerry
> >>> >> >
> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >>> >> > <ui...@gmail.com>
> >>> >> wrote:
> >>> >> >
> >>> >> >> Not sure what is causing this. I need to stage this with UIMA-AS
> >>> >> >> 2.8.1.
> >>> >> I
> >>> >> >> am currently testing with trunk (latest) code. The most current
> >>> >> >> code
> >>> >> does
> >>> >> >> have newer broker.
> >>> >> >>
> >>> >> >> I've started a broker and a sample service:
> >>> >> >>
> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >>> >> >>
> >>> >> >> After the service started, I stopped the broker, waited a bit,
> and
> >>> >> >> restarted the broker.
> >>> >> >>
> >>> >> >> With UIMA-AS from trunk I see the following in the log:
> >>> >> >>
> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
> >>> >> >> Meeting
> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >>> >> >> queue://MeetingDetectorTaeQueue
> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >>> >> >> tcp://localhost:61616
> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
> >>> >> >> Meeting
> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >>> >> >> queue://MeetingDetectorTaeQueue
> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
> Service:
> >>> >> >> Meeting Detector TAE Runtime Exception
> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> >>> Listener
> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >>> >> >> tcp://localhost:61616
> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >>> >> WARNING:
> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect
> To
> >>> >> >> Broker:
> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING:
> Service:
> >>> >> >> Meeting Detector TAE Runtime Exception
> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> >>> Listener
> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >>> >> >> tcp://localhost:61616
> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >>> >> WARNING:
> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect
> To
> >>> >> >> Broker:
> >>> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >>> >> WARNING:
> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> Connection
> >>> to
> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >>> >> WARNING:
> >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered
> Connection
> >>> to
> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >>> >> >>
> >>> >> >>
> >>> >> >> I will repeat the above test with v 2.8.1
> >>> >> >>
> >>> >> >> -Jerry
> >>> >> >>
> >>> >> >>
> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >>> >> nelsonrivera12@gmail.com>
> >>> >> >> wrote:
> >>> >> >>
> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >>> >> >>>
> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com
> >:
> >>> >> >>> > Which version of UIMA-AS are you using?
> >>> >> >>> >
> >>> >> >>> > -Jerry
> >>> >> >>> >
> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >>> >> >>> nelsonrivera12@gmail.com>
> >>> >> >>> > wrote:
> >>> >> >>> >
> >>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
> >>> >> >>> >> processing
> >>> >> >>> >> is
> >>> >> >>> >> executed correctly, but when the broker is stopped and
> >>> >> >>> >> subsequently
> >>> >> >>> >> restarted. UIMA-AS log show the lines below and after this
> >>> >> >>> >> the
> >>> >> >>> >> api
> >>> >> >>> >> client uima-as is not notified in the listener when a cas is
> >>> >> >>> >> processed, any more.
> >>> >> >>> >> can help me?:
> >>> >> >>> >>
> >>> >> >>> >> 02:08:25.570 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >>> Exception
> >>> >> >>> >> 02:08:25.570 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>> >> XClusterAnalyzerAggregate
> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>> javax.jms.JMSException:
> >>> >> >>> >> java.io.EOFException
> >>> >> >>> >> 02:08:25.570 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> >> >>> nerContainer.
> >>> >> >>> >> handleListenerSetupFailure:
> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
> >>> >> >>> >> Listener
> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying
> >>> >> >>> >> ...
> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1
> >>> Managed
> >>> >> >>> >> By:
> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >>> >> >>> >> java.io.EOFException
> >>> >> >>> >> 02:09:02.222 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> >> >>> nerContainer.
> >>> >> >>> >> handleListenerSetupFailure:
> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
> >>> >> >>> >> Listener
> >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
> >>> >> >>> >>
> >>> >> >>> >> 02:09:02.317 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >>> Exception
> >>> >> >>> >> 02:09:02.317 - 14:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>> >> XClusterAnalyzerAggregate
> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The
> connection
> >>> is
> >>> >> >>> >> already closed02:09:02.356 - 15:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> >>> Exception
> >>> >> >>> >> 02:09:02.357 - 15:
> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerCont
> >>> >> >>> >> ainer.onException:
> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>> >> XClusterAnalyzerAggregate
> >>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
> >>> >> >>> >>
> >>> >> >>> >
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >
> >>> >>
> >>> >
> >>>
> >>
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
just like that, the parent CAS comes back but the child CAS doesnt.
And the child CAS if processed, because the uima-as log shows the
lines of the other annotators primitive of the flow processing the
child CAS, but not comes back to UIMA-AS client.
uima-as log only shows
"Jms Listener Failed. Endpoint:
temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException:
java.io.EOFException" ........
ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
Managed By: tcp://localhost:61616 Reason:
javax.jms.IllegalStateException: The Consumer is closed..............
The CAS is send to service after restart the Broker, and how say
parent CAS comes back but the child CAS doesnt. And if i try quiesce
the Aggregate Service console shows : "Quiescing UIMA-AS Service.
Remaining Number of CASes to Process:1". I suppose that is the child
CAS.
the client if can send more CASes to the service, but not receive
response, for each CAS send after the one CAS that was send after
restart broker (for which parent CAS comes back but the child CAS
doesnt) not comes back parent CAS or child CAS.
More specific:
scenario 1:
1- Start Broker and UIMA-AS aggregate service. Everything is fine.
2- Restart Broker (without restart the service). UIMA-AS log show :
"Jms Listener Failed." ....
3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
comes back but the child CAS doesnt, to the uima-as client.)
If i try quiesce the Aggregate Service console shows : "Quiescing
UIMA-AS Service. Remaining Number of CASes to Process:1"
scenario 2:
1- Start Broker and UIMA-AS aggregate service. Everything is fine.
2- Restart Broker (without restart the service). UIMA-AS log show :
"Jms Listener Failed." ....
3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
comes back but the child CAS doesnt, to the uima-as client)
4- Client sends another CAS to UIMA-AS aggregate service.(not comes
back parent CAS or child CAS , to the uima-as client)
5- Client sends another CAS to UIMA-AS aggregate service.(not comes
back parent CAS or child CAS , to the uima-as client)
If i try quiesce the Aggregate Service console shows: "Quiescing
UIMA-AS Service. Remaining Number of CASes to Process:5"
2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> When a parent CAS comes back (after broker is restored), can the client
> send more CASes to the service and receive responses? I am trying to
> determine what the real problem is here.
>
> 1) JMS Listener not recovering connection
> 2) UIMA-AS is not handling CAS reply failures when broker connection is
> stale
>
> -jerry
>
> On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <ui...@gmail.com>
> wrote:
>
>> Strange. You say that the parent CAS comes back but the child CAS doesnt.
>> Are there any exceptions in the service log? Perhaps the service is
>> unable
>> to send the child CAS due to missing broker. There should be something in
>> the uima-as log about this.
>>
>> -jerry
>>
>> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
>> <ne...@gmail.com>
>> wrote:
>>
>>> Yes, it is supposed to do the service, given a
>>> CAS it produces a child CAS and and this CAS continue in the flow,
>>> that are other primitive annotators. And I expected that the child CAS
>>> will sent to UIMA-AS client, but not do.
>>>
>>> the scenario:
>>> 1- Start Broker and UIMA-AS aggregate service
>>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>>> everything is fine. The process is correct.
>>>
>>> 3- Restart Broker (without restart the service). UIMA-AS log show :
>>> "Jms Listener Failed." ....
>>> 4- Client sends a CAS to UIMA-AS aggregate service
>>>
>>> UIMA-AS client receive the parent CAS, but not child CAS.
>>>
>>> Thank for you quick reply
>>>
>>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
>>> given a
>>> > CAS it produces child CASes
>>> > which are then sent to UIMA-AS client. Is this what this service is
>>> > supposed to do if the broker is running?
>>> >
>>> > Can you confirm the scenario:
>>> > 1) Client sends a CAS to UIMA-AS aggregate service
>>> > 2) Service receives the CAS
>>> > 3) Broker dies
>>> > 4) Broker is restarted
>>> >
>>> > Does the UIMA-AS client receive the CAS back in a reply when a broker
>>> > is
>>> > restarted?
>>> >
>>> > -Jerry
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>>> nelsonrivera12@gmail.com>
>>> > wrote:
>>> >
>>> >> I will try the variants that you say. I was deepening the problem:
>>> >>
>>> >> I have a Aggregate AE that contains CasMultiplier, the original CAS
>>> >> is finish when it arrives to this CasMultiplier, this generates new
>>> >> CAS and is this CAS that continue in the flow of Aggregate.
>>> >>
>>> >> After restart broker (without restart the service of annotator) and
>>> >> send a CAS to process, the listener associated to the client api
>>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
>>> >> original CAS finalizes, but the new Cas generated in the
>>> >> CasMultiplier, It is not notifies when finalizes.
>>> >>
>>> >> with Aggregate AE simple the log is the same, but how the original
>>> >> CAS
>>> >> is notifies into the listener when finalizes, not have problem
>>> >> because is not expected for another new CAS.
>>> >>
>>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> >> > If possible can you download AMQ 5.14.0 (latest) and use that
>>> >> > broker
>>> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty
>>> bug
>>> >> > in
>>> >> > AMQ related to keep-alive protocol not working. I am not sure if
>>> >> > this
>>> >> > is
>>> >> > the cause but worth trying.
>>> >> >
>>> >> > You can download AMQ from here:
>>> >> > http://activemq.apache.org/download.html
>>> >> >
>>> >> > -Jerry
>>> >> >
>>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
>>> >> > <ui...@gmail.com>
>>> >> wrote:
>>> >> >
>>> >> >> Not sure what is causing this. I need to stage this with UIMA-AS
>>> >> >> 2.8.1.
>>> >> I
>>> >> >> am currently testing with trunk (latest) code. The most current
>>> >> >> code
>>> >> does
>>> >> >> have newer broker.
>>> >> >>
>>> >> >> I've started a broker and a sample service:
>>> >> >>
>>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
>>> >> >> $UIMA_HOME/examples/deploy/as/
>>> >> >> Deploy_MeetingDetectorTAE.xml
>>> >> >>
>>> >> >> After the service started, I stopped the broker, waited a bit, and
>>> >> >> restarted the broker.
>>> >> >>
>>> >> >> With UIMA-AS from trunk I see the following in the log:
>>> >> >>
>>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>>> >> >> Meeting
>>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>> >> >> queue://MeetingDetectorTaeQueue
>>> >> >> Selector: Command=2000 OR Command=2002 Broker:
>>> >> >> tcp://localhost:61616
>>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller:
>>> >> >> Meeting
>>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>>> >> >> queue://MeetingDetectorTaeQueue
>>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>>> >> >> Meeting Detector TAE Runtime Exception
>>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>>> Listener
>>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>> >> >> tcp://localhost:61616
>>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>>> >> WARNING:
>>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>>> >> >> Broker:
>>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>>> >> >> Meeting Detector TAE Runtime Exception
>>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>>> Listener
>>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>>> >> >> tcp://localhost:61616
>>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>>> >> WARNING:
>>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>>> >> >> Broker:
>>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>>> >> WARNING:
>>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection
>>> to
>>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>>> >> WARNING:
>>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection
>>> to
>>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>>> >> >>
>>> >> >>
>>> >> >> I will repeat the above test with v 2.8.1
>>> >> >>
>>> >> >> -Jerry
>>> >> >>
>>> >> >>
>>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>>> >> nelsonrivera12@gmail.com>
>>> >> >> wrote:
>>> >> >>
>>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>>> >> >>>
>>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> >> >>> > Which version of UIMA-AS are you using?
>>> >> >>> >
>>> >> >>> > -Jerry
>>> >> >>> >
>>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>>> >> >>> nelsonrivera12@gmail.com>
>>> >> >>> > wrote:
>>> >> >>> >
>>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
>>> >> >>> >> processing
>>> >> >>> >> is
>>> >> >>> >> executed correctly, but when the broker is stopped and
>>> >> >>> >> subsequently
>>> >> >>> >> restarted. UIMA-AS log show the lines below and after this
>>> >> >>> >> the
>>> >> >>> >> api
>>> >> >>> >> client uima-as is not notified in the listener when a cas is
>>> >> >>> >> processed, any more.
>>> >> >>> >> can help me?:
>>> >> >>> >>
>>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> Exception
>>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> XClusterAnalyzerAggregate
>>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> javax.jms.JMSException:
>>> >> >>> >> java.io.EOFException
>>> >> >>> >> 02:08:25.570 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> >> >>> nerContainer.
>>> >> >>> >> handleListenerSetupFailure:
>>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>>> >> >>> >> Listener
>>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying
>>> >> >>> >> ...
>>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1
>>> Managed
>>> >> >>> >> By:
>>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>> >> >>> >> java.io.EOFException
>>> >> >>> >> 02:09:02.222 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> >> >>> nerContainer.
>>> >> >>> >> handleListenerSetupFailure:
>>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer
>>> >> >>> >> Listener
>>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
>>> >> >>> >>
>>> >> >>> >> 02:09:02.317 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> Exception
>>> >> >>> >> 02:09:02.317 - 14:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> XClusterAnalyzerAggregate
>>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> >>> >> org.apache.activemq.ConnectionClosedException: The connection
>>> is
>>> >> >>> >> already closed02:09:02.356 - 15:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>>> Exception
>>> >> >>> >> 02:09:02.357 - 15:
>>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerCont
>>> >> >>> >> ainer.onException:
>>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> XClusterAnalyzerAggregate
>>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
>>> >> >>> >>
>>> >> >>> >
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >
>>> >>
>>> >
>>>
>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
When a parent CAS comes back (after broker is restored), can the client
send more CASes to the service and receive responses? I am trying to
determine what the real problem is here.
1) JMS Listener not recovering connection
2) UIMA-AS is not handling CAS reply failures when broker connection is
stale
-jerry
On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
> Strange. You say that the parent CAS comes back but the child CAS doesnt.
> Are there any exceptions in the service log? Perhaps the service is unable
> to send the child CAS due to missing broker. There should be something in
> the uima-as log about this.
>
> -jerry
>
> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> Yes, it is supposed to do the service, given a
>> CAS it produces a child CAS and and this CAS continue in the flow,
>> that are other primitive annotators. And I expected that the child CAS
>> will sent to UIMA-AS client, but not do.
>>
>> the scenario:
>> 1- Start Broker and UIMA-AS aggregate service
>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>> everything is fine. The process is correct.
>>
>> 3- Restart Broker (without restart the service). UIMA-AS log show :
>> "Jms Listener Failed." ....
>> 4- Client sends a CAS to UIMA-AS aggregate service
>>
>> UIMA-AS client receive the parent CAS, but not child CAS.
>>
>> Thank for you quick reply
>>
>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
>> given a
>> > CAS it produces child CASes
>> > which are then sent to UIMA-AS client. Is this what this service is
>> > supposed to do if the broker is running?
>> >
>> > Can you confirm the scenario:
>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> > 2) Service receives the CAS
>> > 3) Broker dies
>> > 4) Broker is restarted
>> >
>> > Does the UIMA-AS client receive the CAS back in a reply when a broker is
>> > restarted?
>> >
>> > -Jerry
>> >
>> >
>> >
>> >
>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> nelsonrivera12@gmail.com>
>> > wrote:
>> >
>> >> I will try the variants that you say. I was deepening the problem:
>> >>
>> >> I have a Aggregate AE that contains CasMultiplier, the original CAS
>> >> is finish when it arrives to this CasMultiplier, this generates new
>> >> CAS and is this CAS that continue in the flow of Aggregate.
>> >>
>> >> After restart broker (without restart the service of annotator) and
>> >> send a CAS to process, the listener associated to the client api
>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
>> >> original CAS finalizes, but the new Cas generated in the
>> >> CasMultiplier, It is not notifies when finalizes.
>> >>
>> >> with Aggregate AE simple the log is the same, but how the original CAS
>> >> is notifies into the listener when finalizes, not have problem
>> >> because is not expected for another new CAS.
>> >>
>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> > If possible can you download AMQ 5.14.0 (latest) and use that broker
>> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty
>> bug
>> >> > in
>> >> > AMQ related to keep-alive protocol not working. I am not sure if this
>> >> > is
>> >> > the cause but worth trying.
>> >> >
>> >> > You can download AMQ from here:
>> >> > http://activemq.apache.org/download.html
>> >> >
>> >> > -Jerry
>> >> >
>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> Not sure what is causing this. I need to stage this with UIMA-AS
>> >> >> 2.8.1.
>> >> I
>> >> >> am currently testing with trunk (latest) code. The most current code
>> >> does
>> >> >> have newer broker.
>> >> >>
>> >> >> I've started a broker and a sample service:
>> >> >>
>> >> >> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
>> >> >> Deploy_MeetingDetectorTAE.xml
>> >> >>
>> >> >> After the service started, I stopped the broker, waited a bit, and
>> >> >> restarted the broker.
>> >> >>
>> >> >> With UIMA-AS from trunk I see the following in the log:
>> >> >>
>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> >> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> >> queue://MeetingDetectorTaeQueue
>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>> Listener
>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> >> tcp://localhost:61616
>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >> WARNING:
>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> >> >> Broker:
>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> >> >> Meeting Detector TAE Runtime Exception
>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
>> Listener
>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> >> tcp://localhost:61616
>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >> WARNING:
>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> >> >> Broker:
>> >> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >> WARNING:
>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection
>> to
>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> >> WARNING:
>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection
>> to
>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >> >>
>> >> >>
>> >> >> I will repeat the above test with v 2.8.1
>> >> >>
>> >> >> -Jerry
>> >> >>
>> >> >>
>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> >> nelsonrivera12@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >> >>>
>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >> >>> > Which version of UIMA-AS are you using?
>> >> >>> >
>> >> >>> > -Jerry
>> >> >>> >
>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >> >>> nelsonrivera12@gmail.com>
>> >> >>> > wrote:
>> >> >>> >
>> >> >>> >> When start ActiveMQ and deploys an annotator as service,
>> >> >>> >> processing
>> >> >>> >> is
>> >> >>> >> executed correctly, but when the broker is stopped and
>> >> >>> >> subsequently
>> >> >>> >> restarted. UIMA-AS log show the lines below and after this the
>> >> >>> >> api
>> >> >>> >> client uima-as is not notified in the listener when a cas is
>> >> >>> >> processed, any more.
>> >> >>> >> can help me?:
>> >> >>> >>
>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> Exception
>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> XClusterAnalyzerAggregate
>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.JMSException:
>> >> >>> >> java.io.EOFException
>> >> >>> >> 02:08:25.570 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >> >>> nerContainer.
>> >> >>> >> handleListenerSetupFailure:
>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1
>> Managed
>> >> >>> >> By:
>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> >>> >> java.io.EOFException
>> >> >>> >> 02:09:02.222 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >> >>> nerContainer.
>> >> >>> >> handleListenerSetupFailure:
>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >> >>> >> Established Connection to Broker: tcp://localhost:61616
>> >> >>> >>
>> >> >>> >> 02:09:02.317 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> Exception
>> >> >>> >> 02:09:02.317 - 14:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> XClusterAnalyzerAggregate
>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>> >> org.apache.activemq.ConnectionClosedException: The connection
>> is
>> >> >>> >> already closed02:09:02.356 - 15:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
>> Exception
>> >> >>> >> 02:09:02.357 - 15:
>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerCont
>> >> >>> >> ainer.onException:
>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> XClusterAnalyzerAggregate
>> >> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
>> >> >>> >>
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >
>>
>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Strange. You say that the parent CAS comes back but the child CAS doesnt.
Are there any exceptions in the service log? Perhaps the service is unable
to send the child CAS due to missing broker. There should be something in
the uima-as log about this.
-jerry
On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera <ne...@gmail.com>
wrote:
> Yes, it is supposed to do the service, given a
> CAS it produces a child CAS and and this CAS continue in the flow,
> that are other primitive annotators. And I expected that the child CAS
> will sent to UIMA-AS client, but not do.
>
> the scenario:
> 1- Start Broker and UIMA-AS aggregate service
> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> everything is fine. The process is correct.
>
> 3- Restart Broker (without restart the service). UIMA-AS log show :
> "Jms Listener Failed." ....
> 4- Client sends a CAS to UIMA-AS aggregate service
>
> UIMA-AS client receive the parent CAS, but not child CAS.
>
> Thank for you quick reply
>
> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
> given a
> > CAS it produces child CASes
> > which are then sent to UIMA-AS client. Is this what this service is
> > supposed to do if the broker is running?
> >
> > Can you confirm the scenario:
> > 1) Client sends a CAS to UIMA-AS aggregate service
> > 2) Service receives the CAS
> > 3) Broker dies
> > 4) Broker is restarted
> >
> > Does the UIMA-AS client receive the CAS back in a reply when a broker is
> > restarted?
> >
> > -Jerry
> >
> >
> >
> >
> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <nelsonrivera12@gmail.com
> >
> > wrote:
> >
> >> I will try the variants that you say. I was deepening the problem:
> >>
> >> I have a Aggregate AE that contains CasMultiplier, the original CAS
> >> is finish when it arrives to this CasMultiplier, this generates new
> >> CAS and is this CAS that continue in the flow of Aggregate.
> >>
> >> After restart broker (without restart the service of annotator) and
> >> send a CAS to process, the listener associated to the client api
> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
> >> original CAS finalizes, but the new Cas generated in the
> >> CasMultiplier, It is not notifies when finalizes.
> >>
> >> with Aggregate AE simple the log is the same, but how the original CAS
> >> is notifies into the listener when finalizes, not have problem
> >> because is not expected for another new CAS.
> >>
> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> > If possible can you download AMQ 5.14.0 (latest) and use that broker
> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty bug
> >> > in
> >> > AMQ related to keep-alive protocol not working. I am not sure if this
> >> > is
> >> > the cause but worth trying.
> >> >
> >> > You can download AMQ from here:
> >> > http://activemq.apache.org/download.html
> >> >
> >> > -Jerry
> >> >
> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> >> wrote:
> >> >
> >> >> Not sure what is causing this. I need to stage this with UIMA-AS
> >> >> 2.8.1.
> >> I
> >> >> am currently testing with trunk (latest) code. The most current code
> >> does
> >> >> have newer broker.
> >> >>
> >> >> I've started a broker and a sample service:
> >> >>
> >> >> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
> >> >> Deploy_MeetingDetectorTAE.xml
> >> >>
> >> >> After the service started, I stopped the broker, waited a bit, and
> >> >> restarted the broker.
> >> >>
> >> >> With UIMA-AS from trunk I see the following in the log:
> >> >>
> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >> queue://MeetingDetectorTaeQueue
> >> >> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >> queue://MeetingDetectorTaeQueue
> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> >> >> Meeting Detector TAE Runtime Exception
> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> Listener
> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >> tcp://localhost:61616
> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >> WARNING:
> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
> >> >> Broker:
> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> >> >> Meeting Detector TAE Runtime Exception
> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms
> Listener
> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> >> tcp://localhost:61616
> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >> WARNING:
> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
> >> >> Broker:
> >> >> tcp://localhost:61616 Retrying Until Successful ...
> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >> WARNING:
> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> >> WARNING:
> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> >> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> >>
> >> >>
> >> >> I will repeat the above test with v 2.8.1
> >> >>
> >> >> -Jerry
> >> >>
> >> >>
> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> >> nelsonrivera12@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >> >>>
> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >> >>> > Which version of UIMA-AS are you using?
> >> >>> >
> >> >>> > -Jerry
> >> >>> >
> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >> >>> nelsonrivera12@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> >> When start ActiveMQ and deploys an annotator as service,
> >> >>> >> processing
> >> >>> >> is
> >> >>> >> executed correctly, but when the broker is stopped and
> >> >>> >> subsequently
> >> >>> >> restarted. UIMA-AS log show the lines below and after this the
> >> >>> >> api
> >> >>> >> client uima-as is not notified in the listener when a cas is
> >> >>> >> processed, any more.
> >> >>> >> can help me?:
> >> >>> >>
> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> Exception
> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> XClusterAnalyzerAggregate
> >> >>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >>> >> java.io.EOFException
> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >> >>> nerContainer.
> >> >>> >> handleListenerSetupFailure:
> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1
> Managed
> >> >>> >> By:
> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >>> >> java.io.EOFException
> >> >>> >> 02:09:02.222 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >> >>> nerContainer.
> >> >>> >> handleListenerSetupFailure:
> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >> >>> >> Established Connection to Broker: tcp://localhost:61616
> >> >>> >>
> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> Exception
> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> XClusterAnalyzerAggregate
> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>> >> org.apache.activemq.ConnectionClosedException: The connection is
> >> >>> >> already closed02:09:02.356 - 15:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime
> Exception
> >> >>> >> 02:09:02.357 - 15:
> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerCont
> >> >>> >> ainer.onException:
> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> XClusterAnalyzerAggregate
> >> >>> >> Managed By: tcp://localhost:61616 Reason:
> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
Yes, it is supposed to do the service, given a
CAS it produces a child CAS and and this CAS continue in the flow,
that are other primitive annotators. And I expected that the child CAS
will sent to UIMA-AS client, but not do.
the scenario:
1- Start Broker and UIMA-AS aggregate service
2- Client sends a CAS to UIMA-AS aggregate service to verify
everything is fine. The process is correct.
3- Restart Broker (without restart the service). UIMA-AS log show :
"Jms Listener Failed." ....
4- Client sends a CAS to UIMA-AS aggregate service
UIMA-AS client receive the parent CAS, but not child CAS.
Thank for you quick reply
2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning given a
> CAS it produces child CASes
> which are then sent to UIMA-AS client. Is this what this service is
> supposed to do if the broker is running?
>
> Can you confirm the scenario:
> 1) Client sends a CAS to UIMA-AS aggregate service
> 2) Service receives the CAS
> 3) Broker dies
> 4) Broker is restarted
>
> Does the UIMA-AS client receive the CAS back in a reply when a broker is
> restarted?
>
> -Jerry
>
>
>
>
> On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> I will try the variants that you say. I was deepening the problem:
>>
>> I have a Aggregate AE that contains CasMultiplier, the original CAS
>> is finish when it arrives to this CasMultiplier, this generates new
>> CAS and is this CAS that continue in the flow of Aggregate.
>>
>> After restart broker (without restart the service of annotator) and
>> send a CAS to process, the listener associated to the client api
>> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
>> original CAS finalizes, but the new Cas generated in the
>> CasMultiplier, It is not notifies when finalizes.
>>
>> with Aggregate AE simple the log is the same, but how the original CAS
>> is notifies into the listener when finalizes, not have problem
>> because is not expected for another new CAS.
>>
>> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > If possible can you download AMQ 5.14.0 (latest) and use that broker
>> > instead of the one the uima-as 2.8.1 ships with? There was a nasty bug
>> > in
>> > AMQ related to keep-alive protocol not working. I am not sure if this
>> > is
>> > the cause but worth trying.
>> >
>> > You can download AMQ from here:
>> > http://activemq.apache.org/download.html
>> >
>> > -Jerry
>> >
>> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com>
>> wrote:
>> >
>> >> Not sure what is causing this. I need to stage this with UIMA-AS
>> >> 2.8.1.
>> I
>> >> am currently testing with trunk (latest) code. The most current code
>> does
>> >> have newer broker.
>> >>
>> >> I've started a broker and a sample service:
>> >>
>> >> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
>> >> Deploy_MeetingDetectorTAE.xml
>> >>
>> >> After the service started, I stopped the broker, waited a bit, and
>> >> restarted the broker.
>> >>
>> >> With UIMA-AS from trunk I see the following in the log:
>> >>
>> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> queue://MeetingDetectorTaeQueue
>> >> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
>> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> >> Detector TAE Trying to Start Listener on Endpoint:
>> >> queue://MeetingDetectorTaeQueue
>> >> Selector: Command=2001 Broker: tcp://localhost:61616
>> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> >> Meeting Detector TAE Runtime Exception
>> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
>> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> tcp://localhost:61616
>> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> WARNING:
>> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> >> Broker:
>> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> >> Meeting Detector TAE Runtime Exception
>> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
>> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> >> tcp://localhost:61616
>> >> Reason: javax.jms.JMSException: java.io.EOFException
>> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> WARNING:
>> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> >> Broker:
>> >> tcp://localhost:61616 Retrying Until Successful ...
>> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> WARNING:
>> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
>> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
>> WARNING:
>> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
>> >> Broker: tcp://localhost:61616 - Ready to Process Again
>> >>
>> >>
>> >> I will repeat the above test with v 2.8.1
>> >>
>> >> -Jerry
>> >>
>> >>
>> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
>> nelsonrivera12@gmail.com>
>> >> wrote:
>> >>
>> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>> >>>
>> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> >>> > Which version of UIMA-AS are you using?
>> >>> >
>> >>> > -Jerry
>> >>> >
>> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> >>> nelsonrivera12@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> >> When start ActiveMQ and deploys an annotator as service,
>> >>> >> processing
>> >>> >> is
>> >>> >> executed correctly, but when the broker is stopped and
>> >>> >> subsequently
>> >>> >> restarted. UIMA-AS log show the lines below and after this the
>> >>> >> api
>> >>> >> client uima-as is not notified in the listener when a cas is
>> >>> >> processed, any more.
>> >>> >> can help me?:
>> >>> >>
>> >>> >> 02:08:25.570 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >>> >> 02:08:25.570 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> XClusterAnalyzerAggregate
>> >>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >>> >> java.io.EOFException
>> >>> >> 02:08:25.570 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerContainer.
>> >>> >> handleListenerSetupFailure:
>> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
>> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed
>> >>> >> By:
>> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >>> >> java.io.EOFException
>> >>> >> 02:09:02.222 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> >>> nerContainer.
>> >>> >> handleListenerSetupFailure:
>> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >>> >> Established Connection to Broker: tcp://localhost:61616
>> >>> >>
>> >>> >> 02:09:02.317 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >>> >> 02:09:02.317 - 14:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> XClusterAnalyzerAggregate
>> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>> >> org.apache.activemq.ConnectionClosedException: The connection is
>> >>> >> already closed02:09:02.356 - 15:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >>> >> 02:09:02.357 - 15:
>> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >>> >> ainer.onException:
>> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
>> XClusterAnalyzerAggregate
>> >>> >> Managed By: tcp://localhost:61616 Reason:
>> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning given a
CAS it produces child CASes
which are then sent to UIMA-AS client. Is this what this service is
supposed to do if the broker is running?
Can you confirm the scenario:
1) Client sends a CAS to UIMA-AS aggregate service
2) Service receives the CAS
3) Broker dies
4) Broker is restarted
Does the UIMA-AS client receive the CAS back in a reply when a broker is
restarted?
-Jerry
On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <ne...@gmail.com>
wrote:
> I will try the variants that you say. I was deepening the problem:
>
> I have a Aggregate AE that contains CasMultiplier, the original CAS
> is finish when it arrives to this CasMultiplier, this generates new
> CAS and is this CAS that continue in the flow of Aggregate.
>
> After restart broker (without restart the service of annotator) and
> send a CAS to process, the listener associated to the client api
> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
> original CAS finalizes, but the new Cas generated in the
> CasMultiplier, It is not notifies when finalizes.
>
> with Aggregate AE simple the log is the same, but how the original CAS
> is notifies into the listener when finalizes, not have problem
> because is not expected for another new CAS.
>
> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > If possible can you download AMQ 5.14.0 (latest) and use that broker
> > instead of the one the uima-as 2.8.1 ships with? There was a nasty bug in
> > AMQ related to keep-alive protocol not working. I am not sure if this is
> > the cause but worth trying.
> >
> > You can download AMQ from here: http://activemq.apache.org/download.html
> >
> > -Jerry
> >
> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com>
> wrote:
> >
> >> Not sure what is causing this. I need to stage this with UIMA-AS 2.8.1.
> I
> >> am currently testing with trunk (latest) code. The most current code
> does
> >> have newer broker.
> >>
> >> I've started a broker and a sample service:
> >>
> >> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
> >> Deploy_MeetingDetectorTAE.xml
> >>
> >> After the service started, I stopped the broker, waited a bit, and
> >> restarted the broker.
> >>
> >> With UIMA-AS from trunk I see the following in the log:
> >>
> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> >> Detector TAE Trying to Start Listener on Endpoint:
> >> queue://MeetingDetectorTaeQueue
> >> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> >> Detector TAE Trying to Start Listener on Endpoint:
> >> queue://MeetingDetectorTaeQueue
> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> >> Meeting Detector TAE Runtime Exception
> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> tcp://localhost:61616
> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING:
> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
> >> Broker:
> >> tcp://localhost:61616 Retrying Until Successful ...
> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> >> Meeting Detector TAE Runtime Exception
> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
> >> tcp://localhost:61616
> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING:
> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
> >> Broker:
> >> tcp://localhost:61616 Retrying Until Successful ...
> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING:
> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
> WARNING:
> >> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> >> Broker: tcp://localhost:61616 - Ready to Process Again
> >>
> >>
> >> I will repeat the above test with v 2.8.1
> >>
> >> -Jerry
> >>
> >>
> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <
> nelsonrivera12@gmail.com>
> >> wrote:
> >>
> >>> I'm using UIMA-AS 2.8.1, any idea that what happen?
> >>>
> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> >>> > Which version of UIMA-AS are you using?
> >>> >
> >>> > -Jerry
> >>> >
> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
> >>> nelsonrivera12@gmail.com>
> >>> > wrote:
> >>> >
> >>> >> When start ActiveMQ and deploys an annotator as service, processing
> >>> >> is
> >>> >> executed correctly, but when the broker is stopped and subsequently
> >>> >> restarted. UIMA-AS log show the lines below and after this the api
> >>> >> client uima-as is not notified in the listener when a cas is
> >>> >> processed, any more.
> >>> >> can help me?:
> >>> >>
> >>> >> 02:08:25.570 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >>> >> 02:08:25.570 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> XClusterAnalyzerAggregate
> >>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> >>> >> java.io.EOFException
> >>> >> 02:08:25.570 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerContainer.
> >>> >> handleListenerSetupFailure:
> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed
> >>> >> By:
> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >>> >> java.io.EOFException
> >>> >> 02:09:02.222 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
> >>> nerContainer.
> >>> >> handleListenerSetupFailure:
> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >>> >> Established Connection to Broker: tcp://localhost:61616
> >>> >>
> >>> >> 02:09:02.317 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >>> >> 02:09:02.317 - 14:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> XClusterAnalyzerAggregate
> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>> >> org.apache.activemq.ConnectionClosedException: The connection is
> >>> >> already closed02:09:02.356 - 15:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >>> >> 02:09:02.357 - 15:
> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >>> >> ainer.onException:
> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint:
> XClusterAnalyzerAggregate
> >>> >> Managed By: tcp://localhost:61616 Reason:
> >>> >> javax.jms.IllegalStateException: The Consumer is closed.
> >>> >>
> >>> >
> >>>
> >>
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
I will try the variants that you say. I was deepening the problem:
I have a Aggregate AE that contains CasMultiplier, the original CAS
is finish when it arrives to this CasMultiplier, this generates new
CAS and is this CAS that continue in the flow of Aggregate.
After restart broker (without restart the service of annotator) and
send a CAS to process, the listener associated to the client api
uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the
original CAS finalizes, but the new Cas generated in the
CasMultiplier, It is not notifies when finalizes.
with Aggregate AE simple the log is the same, but how the original CAS
is notifies into the listener when finalizes, not have problem
because is not expected for another new CAS.
2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> If possible can you download AMQ 5.14.0 (latest) and use that broker
> instead of the one the uima-as 2.8.1 ships with? There was a nasty bug in
> AMQ related to keep-alive protocol not working. I am not sure if this is
> the cause but worth trying.
>
> You can download AMQ from here: http://activemq.apache.org/download.html
>
> -Jerry
>
> On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
>
>> Not sure what is causing this. I need to stage this with UIMA-AS 2.8.1. I
>> am currently testing with trunk (latest) code. The most current code does
>> have newer broker.
>>
>> I've started a broker and a sample service:
>>
>> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
>> Deploy_MeetingDetectorTAE.xml
>>
>> After the service started, I stopped the broker, waited a bit, and
>> restarted the broker.
>>
>> With UIMA-AS from trunk I see the following in the log:
>>
>> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
>> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> Detector TAE Trying to Start Listener on Endpoint:
>> queue://MeetingDetectorTaeQueue
>> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
>> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
>> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
>> Detector TAE Trying to Start Listener on Endpoint:
>> queue://MeetingDetectorTaeQueue
>> Selector: Command=2001 Broker: tcp://localhost:61616
>> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> Meeting Detector TAE Runtime Exception
>> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
>> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> tcp://localhost:61616
>> Reason: javax.jms.JMSException: java.io.EOFException
>> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
>> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> Broker:
>> tcp://localhost:61616 Retrying Until Successful ...
>> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
>> Meeting Detector TAE Runtime Exception
>> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
>> Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
>> tcp://localhost:61616
>> Reason: javax.jms.JMSException: java.io.EOFException
>> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
>> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
>> Broker:
>> tcp://localhost:61616 Retrying Until Successful ...
>> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
>> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
>> Broker: tcp://localhost:61616 - Ready to Process Again
>> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
>> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
>> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
>> Broker: tcp://localhost:61616 - Ready to Process Again
>>
>>
>> I will repeat the above test with v 2.8.1
>>
>> -Jerry
>>
>>
>> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <ne...@gmail.com>
>> wrote:
>>
>>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>>>
>>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>>> > Which version of UIMA-AS are you using?
>>> >
>>> > -Jerry
>>> >
>>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>>> nelsonrivera12@gmail.com>
>>> > wrote:
>>> >
>>> >> When start ActiveMQ and deploys an annotator as service, processing
>>> >> is
>>> >> executed correctly, but when the broker is stopped and subsequently
>>> >> restarted. UIMA-AS log show the lines below and after this the api
>>> >> client uima-as is not notified in the listener when a cas is
>>> >> processed, any more.
>>> >> can help me?:
>>> >>
>>> >> 02:08:25.570 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>>> >> 02:08:25.570 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>>> >> java.io.EOFException
>>> >> 02:08:25.570 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerContainer.
>>> >> handleListenerSetupFailure:
>>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
>>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed
>>> >> By:
>>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>>> >> java.io.EOFException
>>> >> 02:09:02.222 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>>> nerContainer.
>>> >> handleListenerSetupFailure:
>>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>>> >> Established Connection to Broker: tcp://localhost:61616
>>> >>
>>> >> 02:09:02.317 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>>> >> 02:09:02.317 - 14:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> org.apache.activemq.ConnectionClosedException: The connection is
>>> >> already closed02:09:02.356 - 15:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>>> >> 02:09:02.357 - 15:
>>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>>> >> ainer.onException:
>>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>>> >> Managed By: tcp://localhost:61616 Reason:
>>> >> javax.jms.IllegalStateException: The Consumer is closed.
>>> >>
>>> >
>>>
>>
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
If possible can you download AMQ 5.14.0 (latest) and use that broker
instead of the one the uima-as 2.8.1 ships with? There was a nasty bug in
AMQ related to keep-alive protocol not working. I am not sure if this is
the cause but worth trying.
You can download AMQ from here: http://activemq.apache.org/download.html
-Jerry
On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <ui...@gmail.com> wrote:
> Not sure what is causing this. I need to stage this with UIMA-AS 2.8.1. I
> am currently testing with trunk (latest) code. The most current code does
> have newer broker.
>
> I've started a broker and a sample service:
>
> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
> Deploy_MeetingDetectorTAE.xml
>
> After the service started, I stopped the broker, waited a bit, and
> restarted the broker.
>
> With UIMA-AS from trunk I see the following in the log:
>
> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> Detector TAE Trying to Start Listener on Endpoint: queue://MeetingDetectorTaeQueue
> Selector: Command=2000 OR Command=2002 Broker: tcp://localhost:61616
> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> SpringContainerDeployer.doStartListeners: INFO: Controller: Meeting
> Detector TAE Trying to Start Listener on Endpoint: queue://MeetingDetectorTaeQueue
> Selector: Command=2001 Broker: tcp://localhost:61616
> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> Meeting Detector TAE Runtime Exception
> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
> Failed. Endpoint: MeetingDetectorTaeQueue Managed By: tcp://localhost:61616
> Reason: javax.jms.JMSException: java.io.EOFException
> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To Broker:
> tcp://localhost:61616 Retrying Until Successful ...
> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.onException: WARNING: Service:
> Meeting Detector TAE Runtime Exception
> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.onException: WARNING: Jms Listener
> Failed. Endpoint: MeetingDetectorTaeQueue Managed By: tcp://localhost:61616
> Reason: javax.jms.JMSException: java.io.EOFException
> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
> Uima AS Service:Meeting Detector TAE Listener Unable To Connect To Broker:
> tcp://localhost:61616 Retrying Until Successful ...
> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> Broker: tcp://localhost:61616 - Ready to Process Again
> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: WARNING:
> Uima AS Service:Meeting Detector TAE Listener Recovered Connection to
> Broker: tcp://localhost:61616 - Ready to Process Again
>
>
> I will repeat the above test with v 2.8.1
>
> -Jerry
>
>
> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> I'm using UIMA-AS 2.8.1, any idea that what happen?
>>
>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
>> > Which version of UIMA-AS are you using?
>> >
>> > -Jerry
>> >
>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <
>> nelsonrivera12@gmail.com>
>> > wrote:
>> >
>> >> When start ActiveMQ and deploys an annotator as service, processing is
>> >> executed correctly, but when the broker is stopped and subsequently
>> >> restarted. UIMA-AS log show the lines below and after this the api
>> >> client uima-as is not notified in the listener when a cas is
>> >> processed, any more.
>> >> can help me?:
>> >>
>> >> 02:08:25.570 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 02:08:25.570 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> java.io.EOFException
>> >> 02:08:25.570 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerContainer.
>> >> handleListenerSetupFailure:
>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> java.io.EOFException
>> >> 02:09:02.222 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe
>> nerContainer.
>> >> handleListenerSetupFailure:
>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> >> Established Connection to Broker: tcp://localhost:61616
>> >>
>> >> 02:09:02.317 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 02:09:02.317 - 14:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason:
>> >> org.apache.activemq.ConnectionClosedException: The connection is
>> >> already closed02:09:02.356 - 15:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> >> 02:09:02.357 - 15:
>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> >> ainer.onException:
>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason:
>> >> javax.jms.IllegalStateException: The Consumer is closed.
>> >>
>> >
>>
>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Not sure what is causing this. I need to stage this with UIMA-AS 2.8.1. I
am currently testing with trunk (latest) code. The most current code does
have newer broker.
I've started a broker and a sample service:
$UIMA_HOME/bin/deployAsyncService.sh
$UIMA_HOME/examples/deploy/as/Deploy_MeetingDetectorTAE.xml
After the service started, I stopped the broker, waited a bit, and
restarted the broker.
With UIMA-AS from trunk I see the following in the log:
04:21:35.496 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: Meeting Detector TAE Trying to Start Listener on
Endpoint: queue://MeetingDetectorTaeQueue Selector: Command=2000 OR
Command=2002 Broker: tcp://localhost:61616
04:21:35.873 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: Meeting Detector TAE Trying to Start Listener on
Endpoint: queue://MeetingDetectorTaeQueue Selector: Command=2001 Broker:
tcp://localhost:61616
04:21:45.621 - 22:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Meeting Detector TAE Runtime Exception
04:21:45.622 - 22:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException: java.io.EOFException
04:21:45.622 - 22:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
Broker: tcp://localhost:61616 Retrying Until Successful ...
04:21:45.638 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Meeting Detector TAE Runtime Exception
04:21:45.639 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: MeetingDetectorTaeQueue Managed By:
tcp://localhost:61616 Reason: javax.jms.JMSException: java.io.EOFException
04:21:45.639 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Meeting Detector TAE Listener Unable To Connect To
Broker: tcp://localhost:61616 Retrying Until Successful ...
04:22:07.439 - 22:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Meeting Detector TAE Listener Recovered Connection
to Broker: tcp://localhost:61616 - Ready to Process Again
04:22:07.454 - 27:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Meeting Detector TAE Listener Recovered Connection
to Broker: tcp://localhost:61616 - Ready to Process Again
I will repeat the above test with v 2.8.1
-Jerry
On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera <ne...@gmail.com>
wrote:
> I'm using UIMA-AS 2.8.1, any idea that what happen?
>
> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> > Which version of UIMA-AS are you using?
> >
> > -Jerry
> >
> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <nelsonrivera12@gmail.com
> >
> > wrote:
> >
> >> When start ActiveMQ and deploys an annotator as service, processing is
> >> executed correctly, but when the broker is stopped and subsequently
> >> restarted. UIMA-AS log show the lines below and after this the api
> >> client uima-as is not notified in the listener when a cas is
> >> processed, any more.
> >> can help me?:
> >>
> >> 02:08:25.570 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 02:08:25.570 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException
> >> 02:08:25.570 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException
> >> 02:09:02.222 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> >> handleListenerSetupFailure:
> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> >> Established Connection to Broker: tcp://localhost:61616
> >>
> >> 02:09:02.317 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 02:09:02.317 - 14:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason:
> >> org.apache.activemq.ConnectionClosedException: The connection is
> >> already closed02:09:02.356 - 15:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> >> 02:09:02.357 - 15:
> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> >> ainer.onException:
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason:
> >> javax.jms.IllegalStateException: The Consumer is closed.
> >>
> >
>
Re: Broken conections after ACTIVEMQ restart
Posted by nelson rivera <ne...@gmail.com>.
I'm using UIMA-AS 2.8.1, any idea that what happen?
2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <ui...@gmail.com>:
> Which version of UIMA-AS are you using?
>
> -Jerry
>
> On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <ne...@gmail.com>
> wrote:
>
>> When start ActiveMQ and deploys an annotator as service, processing is
>> executed correctly, but when the broker is stopped and subsequently
>> restarted. UIMA-AS log show the lines below and after this the api
>> client uima-as is not notified in the listener when a cas is
>> processed, any more.
>> can help me?:
>>
>> 02:08:25.570 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> 02:08:25.570 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 02:08:25.570 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
>> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
>> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
>> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> 02:09:02.222 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
>> handleListenerSetupFailure:
>> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
>> Established Connection to Broker: tcp://localhost:61616
>>
>> 02:09:02.317 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> 02:09:02.317 - 14:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> org.apache.activemq.ConnectionClosedException: The connection is
>> already closed02:09:02.356 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
>> 02:09:02.357 - 15:
>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> ainer.onException:
>> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> Managed By: tcp://localhost:61616 Reason:
>> javax.jms.IllegalStateException: The Consumer is closed.
>>
>
Re: Broken conections after ACTIVEMQ restart
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Which version of UIMA-AS are you using?
-Jerry
On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera <ne...@gmail.com>
wrote:
> When start ActiveMQ and deploys an annotator as service, processing is
> executed correctly, but when the broker is stopped and subsequently
> restarted. UIMA-AS log show the lines below and after this the api
> client uima-as is not notified in the listener when a cas is
> processed, any more.
> can help me?:
>
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 02:09:02.222 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> Established Connection to Broker: tcp://localhost:61616
>
> 02:09:02.317 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:09:02.317 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> org.apache.activemq.ConnectionClosedException: The connection is
> already closed02:09:02.356 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:09:02.357 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed.
>