You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Martynas Jusevičius <ma...@atomgraph.com> on 2024/12/10 20:22:38 UTC

[10.1.4] NIO threads seem to hang inexplicably

Hi,

I recently noticed some weird "gaps" in the logs of my JAX-RS
application running on Jersey running on Tomcat (running on Docker):


15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns

15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns

15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns

15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns


there's nothing executing between those request filters, but there's a
~10 second or longer gap.
Later on it seems to go away:


15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns

15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns

15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
Request URI <https://localhost:4443/ns> has not matched any
lapp:Dataset
15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
Authenticating request URI: https://localhost:4443/ns


What could be the explanation for this? There's no heavy load, there's
a single client writing/reading data to/from the app.


Martynas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: [10.1.4] NIO threads seem to hang inexplicably

Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark and Martynas,

On 12/11/24 3:20 AM, Mark Thomas wrote:
> 
> 
> On 10/12/2024 20:22, Martynas Jusevičius wrote:
>> Hi,
>>
>> I recently noticed some weird "gaps" in the logs of my JAX-RS
>> application running on Jersey running on Tomcat (running on Docker):
>>
>>
>> 15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>>
>> there's nothing executing between those request filters, but there's a
>> ~10 second or longer gap.
>> Later on it seems to go away:
>>
>>
>> 15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>> 15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
>> Request URI <https://localhost:4443/ns> has not matched any
>> lapp:Dataset
>> 15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
>> Authenticating request URI: https://localhost:4443/ns
>>
>>
>> What could be the explanation for this? There's no heavy load, there's
>> a single client writing/reading data to/from the app.
> 
> You need to take a series of thread dumps (I would suggest 3 thread 
> dumps ~2 seconds apart) when this issue is happening.
> 
> As for a guess as to what might be going on, some sort of external 
> lookup (DNS maybe?) that is timing out after ~10 seconds?
> 
> More detailed debug logging in your filters may also help identify what 
> is happening.

Also maybe enable GC logging. It's low-impact, so doing it in production 
isn't an issue. Then get a decent GC log viewer and look at the pauses 
and stuff. 10+ seconds for a GC pause seems high, but it's not unheard of.

What VM are you running, what GC, and how big of a heap?

-chris


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: [10.1.4] NIO threads seem to hang inexplicably

Posted by Mark Thomas <ma...@apache.org>.

On 10/12/2024 20:22, Martynas Jusevičius wrote:
> Hi,
> 
> I recently noticed some weird "gaps" in the logs of my JAX-RS
> application running on Jersey running on Tomcat (running on Docker):
> 
> 
> 15:25:06,912 [http-nio-7070-exec-9] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:25:15,759 [http-nio-7070-exec-9] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 15:25:46,078 [http-nio-7070-exec-3] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:25:57,826 [http-nio-7070-exec-3] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 15:26:28,110 [http-nio-7070-exec-5] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:26:40,966 [http-nio-7070-exec-5] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 15:27:11,162 [http-nio-7070-exec-8] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:27:29,535 [http-nio-7070-exec-8] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 
> there's nothing executing between those request filters, but there's a
> ~10 second or longer gap.
> Later on it seems to go away:
> 
> 
> 15:35:45,965 [http-nio-7070-exec-4] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:35:45,965 [http-nio-7070-exec-4] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 15:37:16,405 [http-nio-7070-exec-7] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:37:16,405 [http-nio-7070-exec-7] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 15:38:16,644 [http-nio-7070-exec-1] DEBUG ApplicationFilter:135 -
> Request URI <https://localhost:4443/ns> has not matched any
> lapp:Dataset
> 15:38:16,645 [http-nio-7070-exec-1] DEBUG AuthenticationFilter:98 -
> Authenticating request URI: https://localhost:4443/ns
> 
> 
> What could be the explanation for this? There's no heavy load, there's
> a single client writing/reading data to/from the app.

You need to take a series of thread dumps (I would suggest 3 thread 
dumps ~2 seconds apart) when this issue is happening.

As for a guess as to what might be going on, some sort of external 
lookup (DNS maybe?) that is timing out after ~10 seconds?

More detailed debug logging in your filters may also help identify what 
is happening.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org