You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Martynas Jusevičius <ma...@atomgraph.com> on 2024/12/10 20:22:38 UTC
[10.1.4] NIO threads seem to hang inexplicably
Hi,
I recently noticed some weird "gaps" in the logs of my JAX-RS
application running on Jersey running on Tomcat (running on Docker):
15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
there's nothing executing between those request filters, but there's a
~10 second or longer gap.
Later on it seems to go away:
15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns
What could be the explanation for this? There's no heavy load, there's
a single client writing/reading data to/from the app.
Martynas
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [10.1.4] NIO threads seem to hang inexplicably
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark and Martynas,
On 12/11/24 3:20 AM, Mark Thomas wrote:
>
>
> On 10/12/2024 20:22, Martynas Jusevičius wrote:
>> Hi,
>>
>> I recently noticed some weird "gaps" in the logs of my JAX-RS
>> application running on Jersey running on Tomcat (running on Docker):
>>
>>
>> 15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>>
>> there's nothing executing between those request filters, but there's a
>> ~10 second or longer gap.
>> Later on it seems to go away:
>>
>>
>> 15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>>
>> What could be the explanation for this? There's no heavy load, there's
>> a single client writing/reading data to/from the app.
>
> You need to take a series of thread dumps (I would suggest 3 thread
> dumps ~2 seconds apart) when this issue is happening.
>
> As for a guess as to what might be going on, some sort of external
> lookup (DNS maybe?) that is timing out after ~10 seconds?
>
> More detailed debug logging in your filters may also help identify what
> is happening.
Also maybe enable GC logging. It's low-impact, so doing it in production
isn't an issue. Then get a decent GC log viewer and look at the pauses
and stuff. 10+ seconds for a GC pause seems high, but it's not unheard of.
What VM are you running, what GC, and how big of a heap?
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [10.1.4] NIO threads seem to hang inexplicably
Posted by Mark Thomas <ma...@apache.org>.
On 10/12/2024 20:22, Martynas Jusevičius wrote:
> Hi,
>
> I recently noticed some weird "gaps" in the logs of my JAX-RS
> application running on Jersey running on Tomcat (running on Docker):
>
>
> 15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
> 15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
> 15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
> 15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
>
> there's nothing executing between those request filters, but there's a
> ~10 second or longer gap.
> Later on it seems to go away:
>
>
> 15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
> 15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
> 15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
>
>
> What could be the explanation for this? There's no heavy load, there's
> a single client writing/reading data to/from the app.
You need to take a series of thread dumps (I would suggest 3 thread
dumps ~2 seconds apart) when this issue is happening.
As for a guess as to what might be going on, some sort of external
lookup (DNS maybe?) that is timing out after ~10 seconds?
More detailed debug logging in your filters may also help identify what
is happening.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org