You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by eurotrans-Verlag <ve...@t-online.de> on 2011/07/03 14:54:35 UTC
Multiple errors (regressions) on Tomcat 7.0.17
Hi all,
today I downloaded the not yet released Tomcat 7.0.17 (downloaded it from
http://people.apache.org/~markt/dev/tomcat-7/v7.0.17/bin/ ) to test it on a
Windows Server 2008 R2 (64 bit) system with Sun Java 1.6.0_26 (64 bit),
using IIS 7.5 and the ISAPI Redirector 1.2.32 (from
http://people.apache.org/~mturk/tomcat-connectors/jk/binaries/windows/ ). I
used the 64 bit versions of Tomcat and the ISAPI redirector.
I noticed some regressions which do not occur when using Tomcat 7.0.16
instead.
1) When declaring the CrawlerSessionManagerValve at Engine level in
server.xml:
<Valve className="org.apache.catalina.valves.CrawlerSessionManagerValve"/>
then I get a NPE when a client makes a request to Tomcat (or to IIS, which
is redirected to Tomcat by the ISAPI redirector):
03.07.2011 01:24:08 org.apache.catalina.connector.CoyoteAdapter service
SCHWERWIEGEND: An exception or error occurred in the container during the
request processing
java.lang.NullPointerException
at
org.apache.catalina.valves.CrawlerSessionManagerValve.invoke(CrawlerSessionM
anagerValve.java:147)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:404)
at
org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:197)
at
org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProt
ocol.java:152)
at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:
1731)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)
at java.lang.Thread.run(Thread.java:662)
This is because the uaPattern field in CrawlerSessionManagerValve is null.
With Tomcat 7.0.16, it works fine.
2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20 which is
included in the windows distribution, I get these errors when a client makes
a request to IIS which is redirected to Tomcat, resulting in 100% CPU usage
by tomcat's process:
03.07.2011 01:26:29 org.apache.tomcat.util.net.AprEndpoint$Poller run
SCHWERWIEGEND: Critical poller failure (restarting poller): [610038] APR
does not understand this error code
Sometimes, the entire Tomcat process crashes with an
EXCEPTION_ACCESS_VIOLATION in tcnative-1.dll:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000001001f43a,
pid=2260, tid=232
#
# JRE version: 6.0_26-b03
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode
windows-amd64 compressed oops)
# Problematic frame:
# C [tcnative-1.dll+0x1f43a]
#
Also, this works fine with Tomcat 7.0.16 (also when using the included
Tomcat Native 1.1.20 and the ISAPI redirector 1.2.32): no crashes or
"critical poller failure" messages. Please note that the errors also occur
on a 32 bit Windows Server 2008 system.
These do not seem to occur with the HTTP APR connector. Also, when using the
AJP BIO instead of AJP APR, they don't occur.
I didn't see any reports for this on the dev list, so I thought I post them
here.
Regards,
Konstantin Preißer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by Mark Thomas <ma...@apache.org>.
On 05/07/2011 10:20, Konstantin Kolinko wrote:
> 2011/7/4 Mark Thomas <ma...@apache.org>:
>> On 04/07/2011 16:56, Mark Thomas wrote:
>>> On 03/07/2011 15:14, eurotrans-Verlag wrote:
>>>>>> 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20
>>>>> which is
>>>>>> included in the windows distribution, I get these errors when a
>>>>> client makes
>>>>>> a request to IIS which is redirected to Tomcat, resulting in 100% CPU
>>>>> usage
>>>>>> by tomcat's process:
>>>>>
>> (...)
>>
>> The problem has been found and fixed. I'm leaning towards stopping the
>> 7.0.17 release and re-starting with a 7.0.18 tag.
>>
>
> I do not quite get the fix.
>
> The fix:
> http://svn.apache.org/viewvc?rev=1142748&view=rev
>
> Apparently the issue was introduced by this commit:
> http://svn.apache.org/viewvc?rev=1135208&view=rev
Not it wasn't. r1135208 is fine. The problem was this commit:
http://svn.apache.org/viewvc?rev=1137184&view=rev
That moved the call to Poller.add() to one consistent place for all
connectors but failed to remove the Poller.add() in the AJP Processor.
> r1142748 does not revert r1135208
Correct. As intended.
> and it either removes a valid piece
> of code or a problem was introduced earlier.
Neither of those statements is correct. r1142748 removes the duplicate
call to the Poller that should have been removed in r1137184. Why Linux
copes with this and Windows crashes I have no idea but r1142748 is the
right fix for this regression.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by Konstantin Kolinko <kn...@gmail.com>.
2011/7/4 Mark Thomas <ma...@apache.org>:
> On 04/07/2011 16:56, Mark Thomas wrote:
>> On 03/07/2011 15:14, eurotrans-Verlag wrote:
>>>>> 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20
>>>> which is
>>>>> included in the windows distribution, I get these errors when a
>>>> client makes
>>>>> a request to IIS which is redirected to Tomcat, resulting in 100% CPU
>>>> usage
>>>>> by tomcat's process:
>>>>
>(...)
>
> The problem has been found and fixed. I'm leaning towards stopping the
> 7.0.17 release and re-starting with a 7.0.18 tag.
>
I do not quite get the fix.
The fix:
http://svn.apache.org/viewvc?rev=1142748&view=rev
Apparently the issue was introduced by this commit:
http://svn.apache.org/viewvc?rev=1135208&view=rev
r1142748 does not revert r1135208 and it either removes a valid piece
of code or a problem was introduced earlier. The removed code
originates from r1003177 (9 months ago),
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/ajp/AjpAprProcessor.java?diff_format=l&r1=1003176&r2=1003177
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by "verlag.preisser@t-online.de" <ve...@t-online.de>.
On 04/07/2011 16:56, Mark Thomas wrote:
> The problem has been found and fixed. I'm leaning towards stopping the
> 7.0.17 release and re-starting with a 7.0.18 tag.
>
> Mark
Thanks a lot!
Regards,
Konstantin Preißer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by Mark Thomas <ma...@apache.org>.
On 04/07/2011 16:56, Mark Thomas wrote:
> On 03/07/2011 15:14, eurotrans-Verlag wrote:
>>>> 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20
>>> which is
>>>> included in the windows distribution, I get these errors when a
>>> client makes
>>>> a request to IIS which is redirected to Tomcat, resulting in 100% CPU
>>> usage
>>>> by tomcat's process:
>>>
>>> Under what circumstances? Some requests, all requests? The APR/native
>>> AJP connector works with mod_jk and mod_proxy_ajp on linux so it looks
>>> like either the ISAPI redirector is sending something unexpected that
>>> isn't handled well or the windows specific code in the APR/native
>>> connector isn't handling something.
>>
>> For this test, I downloaded the Tomcat 7.0.17 (windows 64 bit) and extracted
>> it to a directory. I didn't changed anything in the Tomcat directory (also
>> not the config files). I set the ISAPI Redirector 1.2.32 to redirect all
>> requests to Tomcat on port 8009, using ajp13.
>
> OK. I've set up IIS 7 and Tomcat 7.0.17 and I see the same thing.
>
> There has been some refactoring of the connector code but (famous last
> words) it should have been harmless. I'm looking at this now...
The problem has been found and fixed. I'm leaning towards stopping the
7.0.17 release and re-starting with a 7.0.18 tag.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by Mark Thomas <ma...@apache.org>.
On 03/07/2011 15:14, eurotrans-Verlag wrote:
>>> 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20
>> which is
>>> included in the windows distribution, I get these errors when a
>> client makes
>>> a request to IIS which is redirected to Tomcat, resulting in 100% CPU
>> usage
>>> by tomcat's process:
>>
>> Under what circumstances? Some requests, all requests? The APR/native
>> AJP connector works with mod_jk and mod_proxy_ajp on linux so it looks
>> like either the ISAPI redirector is sending something unexpected that
>> isn't handled well or the windows specific code in the APR/native
>> connector isn't handling something.
>
> For this test, I downloaded the Tomcat 7.0.17 (windows 64 bit) and extracted
> it to a directory. I didn't changed anything in the Tomcat directory (also
> not the config files). I set the ISAPI Redirector 1.2.32 to redirect all
> requests to Tomcat on port 8009, using ajp13.
OK. I've set up IIS 7 and Tomcat 7.0.17 and I see the same thing.
There has been some refactoring of the connector code but (famous last
words) it should have been harmless. I'm looking at this now...
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Multiple errors (regressions) on Tomcat 7.0.17
Posted by eurotrans-Verlag <ve...@t-online.de>.
Hi Mark,
> If you explicitly set crawlerUserAgents, that will be resolved.
Thanks.
>
> > 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20
> which is
> > included in the windows distribution, I get these errors when a
> client makes
> > a request to IIS which is redirected to Tomcat, resulting in 100% CPU
> usage
> > by tomcat's process:
>
> Under what circumstances? Some requests, all requests? The APR/native
> AJP connector works with mod_jk and mod_proxy_ajp on linux so it looks
> like either the ISAPI redirector is sending something unexpected that
> isn't handled well or the windows specific code in the APR/native
> connector isn't handling something.
For this test, I downloaded the Tomcat 7.0.17 (windows 64 bit) and extracted
it to a directory. I didn't changed anything in the Tomcat directory (also
not the config files). I set the ISAPI Redirector 1.2.32 to redirect all
requests to Tomcat on port 8009, using ajp13.
I made this exact request multiple times to IIS:
GET / HTTP/1.1
Host: localhost
Connection: keep-alive
This should be handled by the default Tomcat root webapp (which displays the
Tomcat welcome page).
The first 2-4 requests work (the number is not always the same), but then on
the next request, Tomcat CPU usage goes up to 100% and the client never
receives a response (then Tomcat displays the error "Critical poller
failure" some multiple times).
03.07.2011 16:03:49 org.apache.tomcat.util.net.AprEndpoint$Poller run
SCHWERWIEGEND: Critical poller failure (restarting poller): [610038] APR
does not understand this error code
In the isapi log, following lines appear (log_level=info):
[Sun Jul 03 15:54:44.530 2011] [2908:2664] [info] init_jk::jk_isapi_plugin.c
(2602): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
[Sun Jul 03 15:54:44.546 2011] [2908:2664] [info] init_jk::jk_isapi_plugin.c
(2781): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
[Sun Jul 03 15:54:46.327 2011] [2908:1968] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (1296): wrong message format
0x0000 from 127.0.0.1:8009
[Sun Jul 03 15:54:46.452 2011] [2908:1968] [error]
ajp_get_reply::jk_ajp_common.c (2118): (worker1) Tomcat is down or refused
connection. No response has been sent to the client (yet)
[Sun Jul 03 15:54:46.452 2011] [2908:1968] [info]
ajp_service::jk_ajp_common.c (2607): (worker1) sending request to tomcat
failed (recoverable), because of protocol error (attempt=1)
After this occurs, Tomcat doesn't respond to subsequent requests (sent via
IIS).
Then, on shutdown, Tomcat prints:
03.07.2011 16:07:27 org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping
the Server instance.
03.07.2011 16:07:27 org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler [http-apr-8080]
03.07.2011 16:07:28 org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler [ajp-apr-8009]
03.07.2011 16:07:29 org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service Catalina
03.07.2011 16:07:29 org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler [http-apr-8080]
After those lines are printed, it takes some time, then these lines are
printed (a new line is printed every second until I kill the tomcat
process):
WARNUNG: Acceptor thread [http-apr-8080-Acceptor-0] failed to unlock.
Forcing hard socket shutdown.
03.07.2011 16:08:30 org.apache.tomcat.util.net.AprEndpoint stopInternal
WARNUNG: Acceptor thread [http-apr-8080-Acceptor-0] failed to unlock.
Forcing hard socket shutdown.
03.07.2011 16:08:31 org.apache.tomcat.util.net.AprEndpoint stopInternal
WARNUNG: Acceptor thread [http-apr-8080-Acceptor-0] failed to unlock.
Forcing hard socket shutdown.
03.07.2011 16:08:32 org.apache.tomcat.util.net.AprEndpoint stopInternal
WARNUNG: Acceptor thread [http-apr-8080-Acceptor-0] failed to unlock.
Forcing hard socket shutdown.
When I delete the Tomcat 7.0.17 and download the regular 7.0.16 (and extract
it to the same directory, and don't change any of the Tomcat files), it
works fine.
Regards,
Konstantin Preißer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Multiple errors (regressions) on Tomcat 7.0.17
Posted by Mark Thomas <ma...@apache.org>.
On 03/07/2011 13:54, eurotrans-Verlag wrote:
> Hi all,
>
> today I downloaded the not yet released Tomcat 7.0.17 (downloaded it from
> http://people.apache.org/~markt/dev/tomcat-7/v7.0.17/bin/ ) to test it on a
> Windows Server 2008 R2 (64 bit) system with Sun Java 1.6.0_26 (64 bit),
> using IIS 7.5 and the ISAPI Redirector 1.2.32 (from
> http://people.apache.org/~mturk/tomcat-connectors/jk/binaries/windows/ ). I
> used the 64 bit versions of Tomcat and the ISAPI redirector.
>
> I noticed some regressions which do not occur when using Tomcat 7.0.16
> instead.
>
>
> 1) When declaring the CrawlerSessionManagerValve at Engine level in
> server.xml:
>
> <Valve className="org.apache.catalina.valves.CrawlerSessionManagerValve"/>
>
> then I get a NPE when a client makes a request to Tomcat (or to IIS, which
> is redirected to Tomcat by the ISAPI redirector):
If you explicitly set crawlerUserAgents, that will be resolved.
> 2) When using the ISAPI Redirector 1.2.32 and Tomcat Native 1.1.20 which is
> included in the windows distribution, I get these errors when a client makes
> a request to IIS which is redirected to Tomcat, resulting in 100% CPU usage
> by tomcat's process:
Under what circumstances? Some requests, all requests? The APR/native
AJP connector works with mod_jk and mod_proxy_ajp on linux so it looks
like either the ISAPI redirector is sending something unexpected that
isn't handled well or the windows specific code in the APR/native
connector isn't handling something.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org