You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@tomcat.apache.org by 旭东胡 <xu...@outlook.com> on 2018/04/30 00:48:04 UTC

slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Hi,

I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol connector and OpenSSLImplementation. The issue is that a request would be timeout using apache HttpClient and client certificate after serval previous requests. It also happens with RestAssured and SoapUI. Please note it works fine for first several requests and then failed with timeout.

However, this issue is not observed when JMeter, tried both JAVA and non-JAVA implementation, and insomnia REST client being used. I used a static page to rule out application factors. Also Http11NioProtocol works fine for all above clients. The only thing I changed for Http11NioProtocol is to specify protocol="org.apache.coyote.http11.Http11NioProtocol” instead of protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I have another connector configured not checking client certificate. This one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol being used.

Would you please help to identify if I have anything wrong in my configuration? I tried to set the log level to fine. But I did not find anything useful. Please help.

Thank you in advance.
Hugh

Environment Info

Redhat 7.3
Oracle JAVA: jdk1.8.0_172
native connector: tomcat-native-1.2.16
openssl: openssl-1.1.0h
Tomcat version: apache-tomcat-8.5.30

behavior

http client hangs after serval calls with a valid client certificate. It may get responses after a long delay or report a timeout error

server.xml
https://pastebin.com/y3ndkLzh

Tow connectors were specified in above configuration. The one with problem is listening to port 11443.

my testing code
https://pastebin.com/7U205QS4

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Rémy Maucherat <re...@apache.org>.

On Wed, May 2, 2018 at 12:22 PM Mark Thomas <ma...@apache.org> wrote:

> On 02/05/18 09:31, Rémy Maucherat wrote:
> > On Wed, May 2, 2018 at 5:19 AM Hugh H <xu...@outlook.com> wrote:
> >
> >> Hi Mark,
> >>
> >> Here are the logs you requested
> >>
> >> client:
> >> https://1drv.ms/t/s!Aii8T4l0bnqVlyAuRIjSuluBe8vy
> >>
> >> server:
> >> https://1drv.ms/u/s!Aii8T4l0bnqVlx-TGo6I0dMXZxG1
> >>
> >>
> >> I checked the system clock right before my testing and the server and
> the
> >> client are synchronized.
> >>
> >
> > Ok, so all seems fine with the SSL handshake, reading the request data,
> but
> > then there is some error processing the request. The processor is already
> > recycled and the ERROR status leads to closing.
> >
> > I have no idea what could fail or how since it's a 100% super simple
> > request. Maybe enable FINE for everything and trying to see what happens
> > between these two:
> > 01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
> > org.apache.coyote.AbstractProcessorLight.process Socket:
> > [org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper@ed820bc
> > :org.apache.tomcat.util.net.SecureNio2Channel@2878c1ae:sun.nio.ch
> .UnixAsynchronousSocketChannelImpl[connected
> > local=/1***.***.***.*67:11443 remote=/1**.***.***.*81:24776]], Status in:
> > [OPEN_READ], State out: [OPEN]
> > 01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
> > org.apache.coyote.AbstractProtocol$ConnectionHandler.release Pushed
> > Processor [org.apache.coyote.http11.Http11Processor@739e50bf]
>
> I read those logs slightly differently. The first request appears to be
> processed normally. The "State out: [OPEN]" indicates that.
>
> The processing thread (https-openssl-nio2-11443-exec-5) completes
> normally. There is no pipelined data else it would be processed on this
> same thread.
>
> The next log message is on a new thread:
>
> 01-May-2018 15:22:25.867 FINE [https-openssl-nio2-11443-exec-6]
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process Processing
> socket
> [org.apache.tomcat.util.net.SecureNio2Channel@2878c1ae:sun.nio.ch
> .UnixAsynchronousSocketChannelImpl[connected
> local=/1***.***.***.*67:11443 remote=/1**.***.***.*81:24776]] with
> status [ERROR]
>
> That indicates to me that at some point during the processing of the
> request, an error occurred and a call was made to endpoint.processSocket
> with SocketEvent.ERROR. Because we only allow a single thread to use a
> socket at a time, this event was queued until the previous thread
> completed. As soon as exec-5 completed, this event was processed on
> exec-6 and closed the socket.
>
> Since the client never receives the response, my guess is that there is
> an I/O error on the write.
>
> I've looked at all the places were SocketEvent.ERROR is triggered. If
> you exclude Async requests, HTTP/2 and WebSocket and anything associated
> with the NIO or APR connectors, the only thing left is the NIO2
> completion handlers. It looks like one of the completion handlers is
> failing. The problem is that we don't know why.
>
> I think it makes sense to add some debug logging to each of the NIO2
> completion handlers that logs the exception in case of a failure.
>
> If I do this, are you able to build Tomcat trunk from source and re-test
> with that?
>

If it is only blocking writes causing an error, then it's not using any
completion handlers, it's the future write in
SecureNio2Channel.FutureWrite. Ok with adding debug logging for the
"fails", it doesn't hurt obviously.

Rémy

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Mark Thomas <ma...@apache.org>.

On 02/05/18 09:31, Rémy Maucherat wrote:
> On Wed, May 2, 2018 at 5:19 AM Hugh H <xu...@outlook.com> wrote:
> 
>> Hi Mark,
>>
>> Here are the logs you requested
>>
>> client:
>> https://1drv.ms/t/s!Aii8T4l0bnqVlyAuRIjSuluBe8vy
>>
>> server:
>> https://1drv.ms/u/s!Aii8T4l0bnqVlx-TGo6I0dMXZxG1
>>
>>
>> I checked the system clock right before my testing and the server and the
>> client are synchronized.
>>
> 
> Ok, so all seems fine with the SSL handshake, reading the request data, but
> then there is some error processing the request. The processor is already
> recycled and the ERROR status leads to closing.
> 
> I have no idea what could fail or how since it's a 100% super simple
> request. Maybe enable FINE for everything and trying to see what happens
> between these two:
> 01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
> org.apache.coyote.AbstractProcessorLight.process Socket:
> [org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper@ed820bc
> :org.apache.tomcat.util.net.SecureNio2Channel@2878c1ae:sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected
> local=/1***.***.***.*67:11443 remote=/1**.***.***.*81:24776]], Status in:
> [OPEN_READ], State out: [OPEN]
> 01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
> org.apache.coyote.AbstractProtocol$ConnectionHandler.release Pushed
> Processor [org.apache.coyote.http11.Http11Processor@739e50bf]

I read those logs slightly differently. The first request appears to be
processed normally. The "State out: [OPEN]" indicates that.

The processing thread (https-openssl-nio2-11443-exec-5) completes
normally. There is no pipelined data else it would be processed on this
same thread.

The next log message is on a new thread:

01-May-2018 15:22:25.867 FINE [https-openssl-nio2-11443-exec-6]
org.apache.coyote.AbstractProtocol$ConnectionHandler.process Processing
socket
[org.apache.tomcat.util.net.SecureNio2Channel@2878c1ae:sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected
local=/1***.***.***.*67:11443 remote=/1**.***.***.*81:24776]] with
status [ERROR]

That indicates to me that at some point during the processing of the
request, an error occurred and a call was made to endpoint.processSocket
with SocketEvent.ERROR. Because we only allow a single thread to use a
socket at a time, this event was queued until the previous thread
completed. As soon as exec-5 completed, this event was processed on
exec-6 and closed the socket.

Since the client never receives the response, my guess is that there is
an I/O error on the write.

I've looked at all the places were SocketEvent.ERROR is triggered. If
you exclude Async requests, HTTP/2 and WebSocket and anything associated
with the NIO or APR connectors, the only thing left is the NIO2
completion handlers. It looks like one of the completion handlers is
failing. The problem is that we don't know why.

I think it makes sense to add some debug logging to each of the NIO2
completion handlers that logs the exception in case of a failure.

If I do this, are you able to build Tomcat trunk from source and re-test
with that?

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Rémy Maucherat <re...@apache.org>.

On Wed, May 2, 2018 at 5:19 AM Hugh H <xu...@outlook.com> wrote:

> Hi Mark,
>
> Here are the logs you requested
>
> client:
> https://1drv.ms/t/s!Aii8T4l0bnqVlyAuRIjSuluBe8vy
>
> server:
> https://1drv.ms/u/s!Aii8T4l0bnqVlx-TGo6I0dMXZxG1
>
>
> I checked the system clock right before my testing and the server and the
> client are synchronized.
>

Ok, so all seems fine with the SSL handshake, reading the request data, but
then there is some error processing the request. The processor is already
recycled and the ERROR status leads to closing.

I have no idea what could fail or how since it's a 100% super simple
request. Maybe enable FINE for everything and trying to see what happens
between these two:
01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
org.apache.coyote.AbstractProcessorLight.process Socket:
[org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper@ed820bc
:org.apache.tomcat.util.net.SecureNio2Channel@2878c1ae:sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected
local=/1***.***.***.*67:11443 remote=/1**.***.***.*81:24776]], Status in:
[OPEN_READ], State out: [OPEN]
01-May-2018 15:22:25.854 FINE [https-openssl-nio2-11443-exec-5]
org.apache.coyote.AbstractProtocol$ConnectionHandler.release Pushed
Processor [org.apache.coyote.http11.Http11Processor@739e50bf]

Rémy


>
> Thanks,
> Hugh
>
> On May 1, 2018, at 9:31 AM, Mark Thomas <markt@apache.org<mailto:
> markt@apache.org>> wrote:
>
> On 01/05/18 03:11, 旭东 胡 wrote:
> Hi Mark,
>
> Unfortunately,  8.5.31 does not resolve my issue. You can find the
> catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo.
> From my client log the timeout occurs:
> 1. between 15:03:48 and 15:04:48
> 2. between 15:04:48 and 15:05:48
> 3. between 15:05:49 and 15:06:49
> 4. between 15:06:59 and 15:07:49
> 5. between 15:07:59 and 15:08:49
> 6. between 15:08:59 and 15:09:49
>
> The problematic port is 11443. Sorry there is a health checking, which I
> cannot turned off, on port 10443 adding a lot noise.
>
> OK.
>
> First of all, please ensure that the time on the client and server are
> synchronized. Give that the server log doesn't show the server starting
> until 15:04:00 the client and server look to be ~25 seconds out of sync.
>
> What I see in most of the connections is the TLS handshake completing
> and the I/O layer passing the socket to the protocol layer for
> processing. The socket is returned from the protocol layer with an
> instruction to close the socket.
>
> We need to see what is happening in the protocol layer. Please add the
> following to logging.properties, restart Tomcat 8.5.31 and repeat your
> test:
> org.apache.coyote.level=FINE
>
> Please also include the client logs this time.
>
> Thanks,
>
> Mark
>
>
>
> Thanks,
> Hugh
>
> On Apr 30, 2018, at 5:08 AM, Mark Thomas <markt@apache.org<mailto:
> markt@apache.org>> wrote:
>
> On 30/04/18 01:48, ** * wrote:
> Hi,
>
> I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol
> connector and OpenSSLImplementation. The issue is that a request would be
> timeout using apache HttpClient and client certificate after serval
> previous requests. It also happens with RestAssured and SoapUI. Please note
> it works fine for first several requests and then failed with timeout.
>
> However, this issue is not observed when JMeter, tried both JAVA and
> non-JAVA implementation, and insomnia REST client being used. I used a
> static page to rule out application factors. Also Http11NioProtocol works
> fine for all above clients. The only thing I changed for Http11NioProtocol
> is to specify  protocol="org.apache.coyote.http11.Http11NioProtocol”
> instead of protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I
> have another  connector configured not checking client certificate. This
> one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol
> being used.
>
> Would you please help to identify if I have anything wrong in my
> configuration?  I tried to set the log level to fine. But I did not find
> anything useful. Please help.
>
> 8.5.31 fixes an error in this area that might be relevant. The release
> vote for 8.5.31 is currently in progress. Details on the dev@ list. If
> you could download the 8.5.31 release candidate and test against that,
> that would be helpful.
>
> If that doesn't work then we'll need the following (again with 8.5.31 so
> we are testing the latest code):
>
> Enable debug logging for the I/O layer:
> org.apache.tomcat.util.net.level=FINE
>
> Enable TLS debug logging for the client:
> -Djavax.net.debug=all
>
> Recreate the problem.
>
> Provide us with:
> - the logs for the 30s before the error and 5s after it
> - the point in the logs where the error occurred
>
> Thanks,
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<mailto:
> users-unsubscribe@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org<mailto:
> users-help@tomcat.apache.org>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<mailto:
> users-unsubscribe@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org<mailto:
> users-help@tomcat.apache.org>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<mailto:
> users-unsubscribe@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org<mailto:
> users-help@tomcat.apache.org>
>
>

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Hugh H <xu...@outlook.com>.

Hi Mark,

Here are the logs you requested

client:
https://1drv.ms/t/s!Aii8T4l0bnqVlyAuRIjSuluBe8vy

server:
https://1drv.ms/u/s!Aii8T4l0bnqVlx-TGo6I0dMXZxG1

I checked the system clock right before my testing and the server and the client are synchronized.

Thanks,
Hugh

On May 1, 2018, at 9:31 AM, Mark Thomas <ma...@apache.org>> wrote:

On 01/05/18 03:11, 旭东 胡 wrote:
Hi Mark,

Unfortunately, 8.5.31 does not resolve my issue. You can find the catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo.
From my client log the timeout occurs:
1. between 15:03:48 and 15:04:48
2. between 15:04:48 and 15:05:48
3. between 15:05:49 and 15:06:49
4. between 15:06:59 and 15:07:49
5. between 15:07:59 and 15:08:49
6. between 15:08:59 and 15:09:49

The problematic port is 11443. Sorry there is a health checking, which I cannot turned off, on port 10443 adding a lot noise.

OK.

First of all, please ensure that the time on the client and server are
synchronized. Give that the server log doesn't show the server starting
until 15:04:00 the client and server look to be ~25 seconds out of sync.

What I see in most of the connections is the TLS handshake completing
and the I/O layer passing the socket to the protocol layer for
processing. The socket is returned from the protocol layer with an
instruction to close the socket.

We need to see what is happening in the protocol layer. Please add the
following to logging.properties, restart Tomcat 8.5.31 and repeat your test:
org.apache.coyote.level=FINE

Please also include the client logs this time.

Thanks,

Mark

Thanks,
Hugh

On Apr 30, 2018, at 5:08 AM, Mark Thomas <ma...@apache.org>> wrote:

On 30/04/18 01:48, ** * wrote:
Hi,

Would you please help to identify if I have anything wrong in my configuration? I tried to set the log level to fine. But I did not find anything useful. Please help.

8.5.31 fixes an error in this area that might be relevant. The release
vote for 8.5.31 is currently in progress. Details on the dev@ list. If
you could download the 8.5.31 release candidate and test against that,
that would be helpful.

If that doesn't work then we'll need the following (again with 8.5.31 so
we are testing the latest code):

Enable debug logging for the I/O layer:
org.apache.tomcat.util.net.level=FINE

Enable TLS debug logging for the client:
-Djavax.net.debug=all

Recreate the problem.

Provide us with:
- the logs for the 30s before the error and 5s after it
- the point in the logs where the error occurred

Thanks,

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<ma...@tomcat.apache.org>
For additional commands, e-mail: users-help@tomcat.apache.org<ma...@tomcat.apache.org>

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Rémy Maucherat <re...@apache.org>.

On Tue, May 1, 2018 at 3:31 PM Mark Thomas <ma...@apache.org> wrote:

> On 01/05/18 03:11, 旭东 胡 wrote:
> > Hi Mark,
> >
> > Unfortunately,  8.5.31 does not resolve my issue. You can find the
> catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo.
> > From my client log the timeout occurs:
> > 1. between 15:03:48 and 15:04:48
> > 2. between 15:04:48 and 15:05:48
> > 3. between 15:05:49 and 15:06:49
> > 4. between 15:06:59 and 15:07:49
> > 5. between 15:07:59 and 15:08:49
> > 6. between 15:08:59 and 15:09:49
> >
> > The problematic port is 11443. Sorry there is a health checking, which I
> cannot turned off, on port 10443 adding a lot noise.
>
> OK.
>
> First of all, please ensure that the time on the client and server are
> synchronized. Give that the server log doesn't show the server starting
> until 15:04:00 the client and server look to be ~25 seconds out of sync.
>
> What I see in most of the connections is the TLS handshake completing
> and the I/O layer passing the socket to the protocol layer for
> processing. The socket is returned from the protocol layer with an
> instruction to close the socket.
>
> We need to see what is happening in the protocol layer. Please add the
> following to logging.properties, restart Tomcat 8.5.31 and repeat your
> test:
> org.apache.coyote.level=FINE
>
> Please also include the client logs this time.
>

Maybe test with JSSE as well instead of OpenSSL ?

Rémy

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Mark Thomas <ma...@apache.org>.

On 01/05/18 03:11, 旭东 胡 wrote:
> Hi Mark,
> 
> Unfortunately,  8.5.31 does not resolve my issue. You can find the catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo. 
> From my client log the timeout occurs:
> 1. between 15:03:48 and 15:04:48
> 2. between 15:04:48 and 15:05:48
> 3. between 15:05:49 and 15:06:49
> 4. between 15:06:59 and 15:07:49
> 5. between 15:07:59 and 15:08:49
> 6. between 15:08:59 and 15:09:49
> 
> The problematic port is 11443. Sorry there is a health checking, which I cannot turned off, on port 10443 adding a lot noise.

OK.

First of all, please ensure that the time on the client and server are
synchronized. Give that the server log doesn't show the server starting
until 15:04:00 the client and server look to be ~25 seconds out of sync.

What I see in most of the connections is the TLS handshake completing
and the I/O layer passing the socket to the protocol layer for
processing. The socket is returned from the protocol layer with an
instruction to close the socket.

We need to see what is happening in the protocol layer. Please add the
following to logging.properties, restart Tomcat 8.5.31 and repeat your test:
org.apache.coyote.level=FINE

Please also include the client logs this time.

Thanks,

Mark


> 
> Thanks,
> Hugh
> 
>> On Apr 30, 2018, at 5:08 AM, Mark Thomas <ma...@apache.org> wrote:
>>
>> On 30/04/18 01:48, ** * wrote:
>>> Hi,
>>>
>>> I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol connector and OpenSSLImplementation. The issue is that a request would be timeout using apache HttpClient and client certificate after serval previous requests. It also happens with RestAssured and SoapUI. Please note it works fine for first several requests and then failed with timeout.
>>>
>>> However, this issue is not observed when JMeter, tried both JAVA and non-JAVA implementation, and insomnia REST client being used. I used a static page to rule out application factors. Also Http11NioProtocol works fine for all above clients. The only thing I changed for Http11NioProtocol is to specify  protocol="org.apache.coyote.http11.Http11NioProtocol” instead of  protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I have another  connector configured not checking client certificate. This one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol being used.
>>>
>>> Would you please help to identify if I have anything wrong in my configuration?  I tried to set the log level to fine. But I did not find anything useful. Please help.
>>
>> 8.5.31 fixes an error in this area that might be relevant. The release
>> vote for 8.5.31 is currently in progress. Details on the dev@ list. If
>> you could download the 8.5.31 release candidate and test against that,
>> that would be helpful.
>>
>> If that doesn't work then we'll need the following (again with 8.5.31 so
>> we are testing the latest code):
>>
>> Enable debug logging for the I/O layer:
>> org.apache.tomcat.util.net.level=FINE
>>
>> Enable TLS debug logging for the client:
>> -Djavax.net.debug=all
>>
>> Recreate the problem.
>>
>> Provide us with:
>> - the logs for the 30s before the error and 5s after it
>> - the point in the logs where the error occurred
>>
>> Thanks,
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by 旭东胡 <xu...@outlook.com>.

Hi Mark,

Unfortunately,  8.5.31 does not resolve my issue. You can find the catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo. 
From my client log the timeout occurs:
1. between 15:03:48 and 15:04:48
2. between 15:04:48 and 15:05:48
3. between 15:05:49 and 15:06:49
4. between 15:06:59 and 15:07:49
5. between 15:07:59 and 15:08:49
6. between 15:08:59 and 15:09:49

The problematic port is 11443. Sorry there is a health checking, which I cannot turned off, on port 10443 adding a lot noise.

Thanks,
Hugh

> On Apr 30, 2018, at 5:08 AM, Mark Thomas <ma...@apache.org> wrote:
> 
> On 30/04/18 01:48, ** * wrote:
>> Hi,
>> 
>> I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol connector and OpenSSLImplementation. The issue is that a request would be timeout using apache HttpClient and client certificate after serval previous requests. It also happens with RestAssured and SoapUI. Please note it works fine for first several requests and then failed with timeout.
>> 
>> However, this issue is not observed when JMeter, tried both JAVA and non-JAVA implementation, and insomnia REST client being used. I used a static page to rule out application factors. Also Http11NioProtocol works fine for all above clients. The only thing I changed for Http11NioProtocol is to specify  protocol="org.apache.coyote.http11.Http11NioProtocol” instead of  protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I have another  connector configured not checking client certificate. This one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol being used.
>> 
>> Would you please help to identify if I have anything wrong in my configuration?  I tried to set the log level to fine. But I did not find anything useful. Please help.
> 
> 8.5.31 fixes an error in this area that might be relevant. The release
> vote for 8.5.31 is currently in progress. Details on the dev@ list. If
> you could download the 8.5.31 release candidate and test against that,
> that would be helpful.
> 
> If that doesn't work then we'll need the following (again with 8.5.31 so
> we are testing the latest code):
> 
> Enable debug logging for the I/O layer:
> org.apache.tomcat.util.net.level=FINE
> 
> Enable TLS debug logging for the client:
> -Djavax.net.debug=all
> 
> Recreate the problem.
> 
> Provide us with:
> - the logs for the 30s before the error and 5s after it
> - the point in the logs where the error occurred
> 
> Thanks,
> 
> Mark
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

Posted by Mark Thomas <ma...@apache.org>.

On 30/04/18 01:48, 旭东 胡 wrote:
> Hi,
> 
> I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol connector and OpenSSLImplementation. The issue is that a request would be timeout using apache HttpClient and client certificate after serval previous requests. It also happens with RestAssured and SoapUI. Please note it works fine for first several requests and then failed with timeout.
> 
> However, this issue is not observed when JMeter, tried both JAVA and non-JAVA implementation, and insomnia REST client being used. I used a static page to rule out application factors. Also Http11NioProtocol works fine for all above clients. The only thing I changed for Http11NioProtocol is to specify  protocol="org.apache.coyote.http11.Http11NioProtocol” instead of  protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I have another  connector configured not checking client certificate. This one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol being used.
> 
> Would you please help to identify if I have anything wrong in my configuration?  I tried to set the log level to fine. But I did not find anything useful. Please help.

8.5.31 fixes an error in this area that might be relevant. The release
vote for 8.5.31 is currently in progress. Details on the dev@ list. If
you could download the 8.5.31 release candidate and test against that,
that would be helpful.

If that doesn't work then we'll need the following (again with 8.5.31 so
we are testing the latest code):

Enable debug logging for the I/O layer:
org.apache.tomcat.util.net.level=FINE

Enable TLS debug logging for the client:
-Djavax.net.debug=all

Recreate the problem.

Provide us with:
- the logs for the 30s before the error and 5s after it
- the point in the logs where the error occurred

Thanks,

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org