You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by Frank Enders <fr...@averbis.com> on 2014/06/10 14:08:13 UTC
Re: Hanging UIMA AS requests
Hi Jerry,
thanks for the reply!
We have this situation when the broker gets terminated while requests
are currently being processed.
Apparently, those requests neither recover after the broker gets
restarted nor time out when UimaAsynchronousEngine.Timeout is reached.
Is this the expected behaviour when a broker is getting terminated
unexpectedly?
Thanks and all the best
Frank
Am 22.04.2014 15:22, schrieb Jaroslaw Cwiklik:
> Hi, do you see any evidence in the service log of any problems? The client
> side will block in sendAndReceive() until a reply comes back from the
> service.
>
> You can attach jconsole to the service to see if it is hung somewhere.
> Please check UIMA-AS README for instructions how to configure service to
> enable
> jmx.
>
> Jerry
>
>
> On Tue, Apr 22, 2014 at 4:38 AM, Frank Enders <fr...@averbis.com>wrote:
>
>> Dear all,
>>
>> we are using a synchronous sendAndReceiveCAS() call within a webservice
>> endpoint (JAX WS RI).
>> Doing so, in some cases we find hanging requests, which are not getting
>> completed.
>> I am attaching a corresponding part of a thread dump.
>>
>> We are using UIMA AS 2.4.0. Application environment is Tomcat 6.0.32, JAX
>> WS RI 2.1.7.
>>
>> Have you encountered a similar behaviour?
>>
>> Thank and all the best
>> Frank
>>
>> "catalina-exec-77" Id=3412437 in WAITING cpu=2083520 ms usr=2056580 ms
>> blocked 547742 for -1 ms waited 297560 for -1 ms
>> locks java.util.concurrent.locks.ReentrantLock$NonfairSync@19c88785
>> at sun.misc.Unsafe.park(Native Method)
>> - waiting on (a java.util.concurrent.Semaphore$NonfairSync@3c3939fc)
>> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>> parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>> doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>> acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>> at java.util.concurrent.Semaphore.acquire(Semaphore.java:286)
>> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineComm
>> on_impl.sendAndReceiveCAS(BaseUIMAAsynchronousEngineCommon_impl.java:2062)
>> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineComm
>> on_impl.sendAndReceiveCAS(BaseUIMAAsynchronousEngineCommon_impl.java:1952)
>> at de.averbis.extraction.[...].webservice.WebservicePort.
>> getLanguage(WebservicePort.java:1214)
>> at sun.reflect.GeneratedMethodAccessor2765.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.sun.xml.ws.api.server.InstanceResolver$1.invoke(
>> InstanceResolver.java:246)
>> at com.sun.xml.ws.server.InvokerTube$2.invoke(InvokerTube.java:146)
>> at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(
>> EndpointMethodHandler.java:257)
>> at com.sun.xml.ws.server.sei.SEIInvokerTube.processRequest(
>> SEIInvokerTube.java:93)
>> at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:598)
>> at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:557)
>> at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:542)
>> at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)
>> - locked (a com.sun.xml.ws.api.pipe.Fiber@5a8fe32) index 19 frame
>> com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)
>> at com.sun.xml.ws.server.WSEndpointImpl$2.process(
>> WSEndpointImpl.java:243)
>> at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.
>> handle(HttpAdapter.java:471)
>> at com.sun.xml.ws.transport.http.HttpAdapter.handle(
>> HttpAdapter.java:244)
>> at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(
>> ServletAdapter.java:135)
>> at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.
>> doGet(WSServletDelegate.java:129)
>> at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.
>> doPost(WSServletDelegate.java:160)
>> at com.sun.xml.ws.transport.http.servlet.WSServlet.doPost(
>> WSServlet.java:75)
>> at javax.servlet.http.HttpServlet.service(Unknown Source)
>> at javax.servlet.http.HttpServlet.service(Unknown Source)
>> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown
>> Source)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown
>> Source)
>> at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown
>> Source)
>> at org.apache.catalina.core.StandardContextValve.invoke(Unknown
>> Source)
>> at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source)
>> at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source)
>> at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source)
>> at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source)
>> at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source)
>> at org.apache.coyote.ajp.AjpAprProcessor.process(Unknown Source)
>> at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(Unknown
>> Source)
>> at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(Unknown
>> Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.
>> runTask(ThreadPoolExecutor.java:895)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:918)
>> at java.lang.Thread.run(Thread.java:662)
>>
--
Averbis GmbH
Tennenbacher Straße 11
D-79106 Freiburg
Fon: +49 (0) 761 - 203 976 92
Fax: +49 (0) 761 - 203 976 94
E-Mail: frank.enders@averbis.com
Geschäftsführer: Dr. med. Philipp Daumke, Dr. Kornél Markó
Sitz der Gesellschaft: Freiburg i. Br.
AG Freiburg i. Br., HRB 701080
Re: Hanging UIMA AS requests
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
Frank, yes the System.exit() is a big problem when deploying UIMA-AS in a
shared JVM.
There is a jvm wide setting to disable calling System.exit(). Use
-DdontKill when starting
Tomcat. It is not a solution, just a hack to get around this problem.
I will investigate if this could be removed. If I recall there are two
scenarios where UIMA-AS
decides to call exit():
1) Catching Java Error (OOM for example). This was put it to force JVM exit
on OOM which does not always happen.
2) AE error threshold (defined in deployment descriptor) is reached.
I'v created a JIRA for the future release of UIMA-AS to engineer a better
solution.
My first thought would be to have the UIMA-AS notify the service
wrapper/container
that it wants to shutdown instead of calling System.exit(). The container
would then
initiate a shutdown to clean things up. If you have any comments on this
please
comment here: https://issues.apache.org/jira/browse/UIMA-3905
Just FYI, there is a Release Candidate for UIMA-AS 2.6.0 being voted on
now. If there are
no issues, it will be released soon. There are a number of fixes there to
deal with thread
cleanup.
Jerry
On Wed, Jun 18, 2014 at 10:05 AM, Frank Enders <fr...@averbis.com>
wrote:
> Hi Jerry,
>
> thanks!
>
> When upgrading to 2.4.2 we observed some issues, which lead to a more
> general question.
>
> I suppose those problems are related to the way we deploy our endpoints.
> We wrote wrappers for both, the AMQ broker and UIMA AS endpints
> (RemoteAsyncAE_API), allowing us to run them within a (Tomcat) container.
> This way we distribute several AS endpoint by deploying them across
> multiple containers.
>
> The AMQ Broker, encapsulated within a war file, is being deployed in a
> container as well.
>
> This was needed for an IT environment, in which we are only allowed to
> deploy within containers without starting dedicated JVMs for each endpoint.
>
> Generally, this works quite well. But it seems that undeployment of
> those components, induced by shutting down their container, may result
> in scattered 'hangig' threads (both, on consumer and producer side).
>
> Is UIMA AS principally designed for being deployed within containers? Or
> do you think we will face further issues following this approach?
> I am aksing since in the recent AS sources we found occurrences of
> System.exit(), which is quite fatefull when running within a shared JVM ;)
>
> Thanks and all the best
> Frank
>
> Am 12.06.2014 16:09, schrieb Jaroslaw Cwiklik:
> > All you need is a reference to a thread that called sendAndReceive().
> Once
> > you have it, just call <myThread>.interrupt().
> >
> > I would still like to know more about the timer not working. Is there
> > anything in the log that shows the UIMA-AS timer expiring?
> >
> > I just ran a test and the following is in the log:
> >
> > Jun 12, 2014 10:07:17 AM org.apache.uima.aae.delegate.Delegate$1
> > Delegate.TimerTask.run
> > WARNING: Timeout While Waiting For Reply From
> > Delegate:SlowNoOpAnnotatorQueue1 Process CAS Request Timed Out.
> Configured
> > Reply Window Of 1,000. Cas Reference Id:-26e1fb2b:1469067657a:-7ff3
> > Jun 12, 2014 10:07:17 AM
> > org.apache.uima.adapter.jms.client.ClientServiceDelegate handleError
> > WARNING: Process Timeout - Uima AS Client Didn't Receive Process Reply
> > Within Configured Window Of:1,000 millis
> > Jun 12, 2014 10:07:17 AM
> > org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl
> > notifyOnTimout
> > WARNING: Request To Process Cas Has Timed-out. Service
> > Queue:SlowNoOpAnnotatorQueue1. Broker: tcp://localhost.localdomain:61617
> > Cas Timed-out on host: 192.168.6.65
> >
> > Jerry
> >
> >
> > On Wed, Jun 11, 2014 at 6:30 PM, Frank Enders <fr...@averbis.com>
> > wrote:
> >
> >> Hi Jerry,
> >>
> >> Am 10.06.2014 21:27, schrieb Jaroslaw Cwiklik:
> >>
> >> The 2.4.2 AS code also supports interrupts on a thread stuck in
> >>> sendAndReceive(). You can implement your own
> >>> timer if you like, and if it pops just interrupt the thread and try
> >>> calling
> >>> sendAndReceive() again. The subsequent call
> >>> should block until a new connection is established.
> >>>
> >>
> >> How would I do this? I don't find anything about this in the 2.4.2 docs.
> >>
> >> Thanks!
> >> Frank
> >>
> >
>
>
> --
> Averbis GmbH
> Tennenbacher Straße 11
> D-79106 Freiburg
>
> Fon: +49 (0) 761 - 203 976 92
> Fax: +49 (0) 761 - 203 976 94
> E-Mail: frank.enders@averbis.com
>
> Geschäftsführer: Dr. med. Philipp Daumke, Dr. Kornél Markó
> Sitz der Gesellschaft: Freiburg i. Br.
> AG Freiburg i. Br., HRB 701080
>
Re: Hanging UIMA AS requests
Posted by Frank Enders <fr...@averbis.com>.
Hi Jerry,
thanks!
When upgrading to 2.4.2 we observed some issues, which lead to a more
general question.
I suppose those problems are related to the way we deploy our endpoints.
We wrote wrappers for both, the AMQ broker and UIMA AS endpints
(RemoteAsyncAE_API), allowing us to run them within a (Tomcat) container.
This way we distribute several AS endpoint by deploying them across
multiple containers.
The AMQ Broker, encapsulated within a war file, is being deployed in a
container as well.
This was needed for an IT environment, in which we are only allowed to
deploy within containers without starting dedicated JVMs for each endpoint.
Generally, this works quite well. But it seems that undeployment of
those components, induced by shutting down their container, may result
in scattered 'hangig' threads (both, on consumer and producer side).
Is UIMA AS principally designed for being deployed within containers? Or
do you think we will face further issues following this approach?
I am aksing since in the recent AS sources we found occurrences of
System.exit(), which is quite fatefull when running within a shared JVM ;)
Thanks and all the best
Frank
Am 12.06.2014 16:09, schrieb Jaroslaw Cwiklik:
> All you need is a reference to a thread that called sendAndReceive(). Once
> you have it, just call <myThread>.interrupt().
>
> I would still like to know more about the timer not working. Is there
> anything in the log that shows the UIMA-AS timer expiring?
>
> I just ran a test and the following is in the log:
>
> Jun 12, 2014 10:07:17 AM org.apache.uima.aae.delegate.Delegate$1
> Delegate.TimerTask.run
> WARNING: Timeout While Waiting For Reply From
> Delegate:SlowNoOpAnnotatorQueue1 Process CAS Request Timed Out. Configured
> Reply Window Of 1,000. Cas Reference Id:-26e1fb2b:1469067657a:-7ff3
> Jun 12, 2014 10:07:17 AM
> org.apache.uima.adapter.jms.client.ClientServiceDelegate handleError
> WARNING: Process Timeout - Uima AS Client Didn't Receive Process Reply
> Within Configured Window Of:1,000 millis
> Jun 12, 2014 10:07:17 AM
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl
> notifyOnTimout
> WARNING: Request To Process Cas Has Timed-out. Service
> Queue:SlowNoOpAnnotatorQueue1. Broker: tcp://localhost.localdomain:61617
> Cas Timed-out on host: 192.168.6.65
>
> Jerry
>
>
> On Wed, Jun 11, 2014 at 6:30 PM, Frank Enders <fr...@averbis.com>
> wrote:
>
>> Hi Jerry,
>>
>> Am 10.06.2014 21:27, schrieb Jaroslaw Cwiklik:
>>
>> The 2.4.2 AS code also supports interrupts on a thread stuck in
>>> sendAndReceive(). You can implement your own
>>> timer if you like, and if it pops just interrupt the thread and try
>>> calling
>>> sendAndReceive() again. The subsequent call
>>> should block until a new connection is established.
>>>
>>
>> How would I do this? I don't find anything about this in the 2.4.2 docs.
>>
>> Thanks!
>> Frank
>>
>
--
Averbis GmbH
Tennenbacher Straße 11
D-79106 Freiburg
Fon: +49 (0) 761 - 203 976 92
Fax: +49 (0) 761 - 203 976 94
E-Mail: frank.enders@averbis.com
Geschäftsführer: Dr. med. Philipp Daumke, Dr. Kornél Markó
Sitz der Gesellschaft: Freiburg i. Br.
AG Freiburg i. Br., HRB 701080
Re: Hanging UIMA AS requests
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
All you need is a reference to a thread that called sendAndReceive(). Once
you have it, just call <myThread>.interrupt().
I would still like to know more about the timer not working. Is there
anything in the log that shows the UIMA-AS timer expiring?
I just ran a test and the following is in the log:
Jun 12, 2014 10:07:17 AM org.apache.uima.aae.delegate.Delegate$1
Delegate.TimerTask.run
WARNING: Timeout While Waiting For Reply From
Delegate:SlowNoOpAnnotatorQueue1 Process CAS Request Timed Out. Configured
Reply Window Of 1,000. Cas Reference Id:-26e1fb2b:1469067657a:-7ff3
Jun 12, 2014 10:07:17 AM
org.apache.uima.adapter.jms.client.ClientServiceDelegate handleError
WARNING: Process Timeout - Uima AS Client Didn't Receive Process Reply
Within Configured Window Of:1,000 millis
Jun 12, 2014 10:07:17 AM
org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl
notifyOnTimout
WARNING: Request To Process Cas Has Timed-out. Service
Queue:SlowNoOpAnnotatorQueue1. Broker: tcp://localhost.localdomain:61617
Cas Timed-out on host: 192.168.6.65
Jerry
On Wed, Jun 11, 2014 at 6:30 PM, Frank Enders <fr...@averbis.com>
wrote:
> Hi Jerry,
>
> Am 10.06.2014 21:27, schrieb Jaroslaw Cwiklik:
>
> The 2.4.2 AS code also supports interrupts on a thread stuck in
>> sendAndReceive(). You can implement your own
>> timer if you like, and if it pops just interrupt the thread and try
>> calling
>> sendAndReceive() again. The subsequent call
>> should block until a new connection is established.
>>
>
> How would I do this? I don't find anything about this in the 2.4.2 docs.
>
> Thanks!
> Frank
>
Re: Hanging UIMA AS requests
Posted by Frank Enders <fr...@averbis.com>.
Hi Jerry,
Am 10.06.2014 21:27, schrieb Jaroslaw Cwiklik:
> The 2.4.2 AS code also supports interrupts on a thread stuck in
> sendAndReceive(). You can implement your own
> timer if you like, and if it pops just interrupt the thread and try calling
> sendAndReceive() again. The subsequent call
> should block until a new connection is established.
How would I do this? I don't find anything about this in the 2.4.2 docs.
Thanks!
Frank
Re: Hanging UIMA AS requests
Posted by Jaroslaw Cwiklik <ui...@gmail.com>.
If the broker dies you will loose all CASes sent to a service. If a service
fails to find a reply queue (broker is gone) it
will just drop the reply, log a message and move on. The temp queue is gone
and even if you restart the broker
the temp queue will not be recovered. The UIMA-AS (both the service and
client) are designed to recover from broker failures
and any new work (after broker restart) will be proceed.
On the UIMA-AS client side make sure you setup a Process timeout. I am not
aware of any issues regarding the client
not timing out if the timeout is used. Is there evidence in the client log
that the internal UIMA-AS timer popped?
The 2.4.2 AS code also supports interrupts on a thread stuck in
sendAndReceive(). You can implement your own
timer if you like, and if it pops just interrupt the thread and try calling
sendAndReceive() again. The subsequent call
should block until a new connection is established.
Jerry
On Tue, Jun 10, 2014 at 8:08 AM, Frank Enders <fr...@averbis.com>
wrote:
> Hi Jerry,
>
> thanks for the reply!
>
> We have this situation when the broker gets terminated while requests are
> currently being processed.
> Apparently, those requests neither recover after the broker gets restarted
> nor time out when UimaAsynchronousEngine.Timeout is reached.
>
> Is this the expected behaviour when a broker is getting terminated
> unexpectedly?
>
> Thanks and all the best
> Frank
>
> Am 22.04.2014 15:22, schrieb Jaroslaw Cwiklik:
>
> Hi, do you see any evidence in the service log of any problems? The client
>> side will block in sendAndReceive() until a reply comes back from the
>> service.
>>
>> You can attach jconsole to the service to see if it is hung somewhere.
>> Please check UIMA-AS README for instructions how to configure service to
>> enable
>> jmx.
>>
>> Jerry
>>
>>
>> On Tue, Apr 22, 2014 at 4:38 AM, Frank Enders <fr...@averbis.com>
>> wrote:
>>
>> Dear all,
>>>
>>> we are using a synchronous sendAndReceiveCAS() call within a webservice
>>> endpoint (JAX WS RI).
>>> Doing so, in some cases we find hanging requests, which are not getting
>>> completed.
>>> I am attaching a corresponding part of a thread dump.
>>>
>>> We are using UIMA AS 2.4.0. Application environment is Tomcat 6.0.32, JAX
>>> WS RI 2.1.7.
>>>
>>> Have you encountered a similar behaviour?
>>>
>>> Thank and all the best
>>> Frank
>>>
>>> "catalina-exec-77" Id=3412437 in WAITING cpu=2083520 ms usr=2056580 ms
>>> blocked 547742 for -1 ms waited 297560 for -1 ms
>>> locks java.util.concurrent.locks.ReentrantLock$NonfairSync@19c88785
>>> at sun.misc.Unsafe.park(Native Method)
>>> - waiting on (a java.util.concurrent.Semaphore$NonfairSync@3c3939fc
>>> )
>>> at java.util.concurrent.locks.LockSupport.park(LockSupport.
>>> java:156)
>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>>> parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>>> doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
>>> acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>>> at java.util.concurrent.Semaphore.acquire(Semaphore.java:286)
>>> at org.apache.uima.adapter.jms.client.
>>> BaseUIMAAsynchronousEngineComm
>>> on_impl.sendAndReceiveCAS(BaseUIMAAsynchronousEngineComm
>>> on_impl.java:2062)
>>> at org.apache.uima.adapter.jms.client.
>>> BaseUIMAAsynchronousEngineComm
>>> on_impl.sendAndReceiveCAS(BaseUIMAAsynchronousEngineComm
>>> on_impl.java:1952)
>>> at de.averbis.extraction.[...].webservice.WebservicePort.
>>> getLanguage(WebservicePort.java:1214)
>>> at sun.reflect.GeneratedMethodAccessor2765.invoke(Unknown Source)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at com.sun.xml.ws.api.server.InstanceResolver$1.invoke(
>>> InstanceResolver.java:246)
>>> at com.sun.xml.ws.server.InvokerTube$2.invoke(InvokerTube.java:146)
>>> at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(
>>> EndpointMethodHandler.java:257)
>>> at com.sun.xml.ws.server.sei.SEIInvokerTube.processRequest(
>>> SEIInvokerTube.java:93)
>>> at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:598)
>>> at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:557)
>>> at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:542)
>>> at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)
>>> - locked (a com.sun.xml.ws.api.pipe.Fiber@5a8fe32) index 19 frame
>>> com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)
>>> at com.sun.xml.ws.server.WSEndpointImpl$2.process(
>>> WSEndpointImpl.java:243)
>>> at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.
>>> handle(HttpAdapter.java:471)
>>> at com.sun.xml.ws.transport.http.HttpAdapter.handle(
>>> HttpAdapter.java:244)
>>> at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(
>>> ServletAdapter.java:135)
>>> at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.
>>> doGet(WSServletDelegate.java:129)
>>> at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.
>>> doPost(WSServletDelegate.java:160)
>>> at com.sun.xml.ws.transport.http.servlet.WSServlet.doPost(
>>> WSServlet.java:75)
>>> at javax.servlet.http.HttpServlet.service(Unknown Source)
>>> at javax.servlet.http.HttpServlet.service(Unknown Source)
>>> at org.apache.catalina.core.ApplicationFilterChain.
>>> internalDoFilter(Unknown
>>> Source)
>>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown
>>> Source)
>>> at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown
>>> Source)
>>> at org.apache.catalina.core.StandardContextValve.invoke(Unknown
>>> Source)
>>> at org.apache.catalina.core.StandardHostValve.invoke(Unknown
>>> Source)
>>> at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown
>>> Source)
>>> at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source)
>>> at org.apache.catalina.core.StandardEngineValve.invoke(Unknown
>>> Source)
>>> at org.apache.catalina.connector.CoyoteAdapter.service(Unknown
>>> Source)
>>> at org.apache.coyote.ajp.AjpAprProcessor.process(Unknown Source)
>>> at org.apache.coyote.ajp.AjpAprProtocol$
>>> AjpConnectionHandler.process(Unknown
>>> Source)
>>> at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.
>>> run(Unknown
>>> Source)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.
>>> runTask(ThreadPoolExecutor.java:895)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>> ThreadPoolExecutor.java:918)
>>> at java.lang.Thread.run(Thread.java:662)
>>>
>>>
>
> --
> Averbis GmbH
> Tennenbacher Straße 11
> D-79106 Freiburg
>
> Fon: +49 (0) 761 - 203 976 92
> Fax: +49 (0) 761 - 203 976 94
> E-Mail: frank.enders@averbis.com
>
> Geschäftsführer: Dr. med. Philipp Daumke, Dr. Kornél Markó
> Sitz der Gesellschaft: Freiburg i. Br.
> AG Freiburg i. Br., HRB 701080
>
>