You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Konstantin Kolinko <kn...@gmail.com> on 2018/12/01 10:00:53 UTC

Re: Tomcat occasionally returns a response without HTTP headers

сб, 1 дек. 2018 г. в 01:30, Kohei Nozaki <ko...@nozaki.me>:
>
> Hello Christopher, thank you for your help.
>
> * Our downstream Nginx instance (The client of our Tomcat instance) recorded the error "upstream sent no valid HTTP/1.0 header while reading response header from upstream" at that time and the error makes perfect sense concerning the response which has neither HTTP status line nor HTTP headers.
>
> Speaking of a possibility of a bug, a person commented on the Stackoverflow question and said that there might be something in the request that possibly downgrades the connection to HTTP/0.9. Do you think it's possible? The comments can be seen from the URL below:
>
> https://stackoverflow.com/questions/53496598/tomcat-occasionally-returns-a-response-without-http-headers#comment93976313_53552752

1. See the official FAQ / Troubleshoting page:
https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics

Especially pay attention to
1) configuring an access log
2) setting system property
org.apache.catalina.connector.RECYCLE_FACADES=true

(and Java ImageIO stream handling bug)

2. HTTP 0.9 is a valid response format (a feature, not a bug) that
does not contain status line nor headers, sends just the requested
document and closes the connection afterwards. The "Specifications"
page in the wiki has a link to the original specification, if you are
interested.

But HTTP 0.9 should never mix itself with chunked encoding. (The "5d +
CRLF" chunk size that you are seeing).

3. If somebody calls "out.flush()" (or response.flushBuffer()) before
writing a response body, the headers and the body will be sent as
separate packets and may appear separately in wireshark.  It is a
valid behaviour.

(But your client should see the headers. It shouldn't report that
headers are missing).

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat occasionally returns a response without HTTP headers

Posted by Mark Thomas <ma...@apache.org>.
On 16/12/2018 03:01, Kohei Nozaki wrote:
> Hello,
> 
> It turned out that the "sjsxp" library which JAX-WS RI v2.1.3 uses makes Tomcat behave this way. I tried a different version of JAX-WS RI (v2.1.7) which doesn't use the "sjsxp" library anymore and it solved the issue.

Thanks for reporting back when you found a solution.

> A very similar issue posted on the Metro mailing list: http://metro.1045641.n5.nabble.com/JAX-WS-RI-2-1-5-returning-malformed-response-tp1063518.html
> 
> It's surprising that a bug of a framework which is built on the Servlet API can make such an issue happen, but anyway thank you very much everyone who helped me out.

I'm guessing it held onto a reference and re-used it when it shouldn't.
That can cause all sorts of chaos.

Mark

> 
> Regards,
> Kohei
> 
>> On Dec 4, 2018, at 3:39, Christopher Schultz <ch...@christopherschultz.net> wrote:
>>
> Kohei,
> 
> On 12/1/18 10:06, Kohei Nozaki wrote:
>>>> Hello Konstantin, thanks for sharing the valuable information.
>>>>
>>>>> On Dec 1, 2018, at 19:00, Konstantin Kolinko
>>>>> <kn...@gmail.com> wrote:
>>>>>
>>>>>> * Our downstream Nginx instance (The client of our Tomcat
>>>>>> instance) recorded the error "upstream sent no valid HTTP/1.0
>>>>>> header while reading response header from upstream" at that
>>>>>> time and the error makes perfect sense concerning the response
>>>>>> which has neither HTTP status line nor HTTP headers.
>>>>>
>>>>> 1. See the official FAQ / Troubleshoting page: 
>>>>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics
>>>>>
>>>>>
>>>>>
> Especially pay attention to
>>>>> 1) configuring an access log 2) setting system property 
>>>>> org.apache.catalina.connector.RECYCLE_FACADES=true
>>>>
>>>> I've investigated "RECYCLE_FACADES" and understand that an
>>>> improperly implemented application which keeps a reference to
>>>> Request or Response objects outside of their lifecycle can make
>>>> such an issue like mine happen (please correct me if I'm wrong..).
>>>>
>>>>
>>>> But I still don't quite understand what does "RECYCLE_FACADES=true"
>>>> do. The Wiki page says "This makes it easier to spot illegal access
>>>> when it happens, instead of waiting until side effects of such
>>>> access become visible" but how does it make easier? Does this
>>>> property make Tomcat produce an Exception or make Tomcat produce
>>>> some warning message to a log file or something when such accesses
>>>> happen, for example?
> 
> Tomcat usually handles requests something like this. Imagine a
> single-threaded server where Tomcat only accepts a single connection
> at a time (just to simplify the code to the point where it fits into a
> ML post).
> 
> Many of these methods are made-up. There is no
> TomcatHttpServletRequest class or a .setRequestLine method in it
> (though there are ... siilar concepts in there, way down deep). The
> point is how the objects are used, or rather *re* used.
> 
> HttpServletRequest request = new TomcatHttpServletRequest();
> HttpServletResponse response = new TomcatHttpServletResponse();
> 
> Connection conn = null;
> 
> while(null != (conn = socket.acceptConnection()) {
>    request.setRequestLine(conn.getRequestLine());
>    request.setInputStream(conn.getInputStream());
>    response.setOutputStream(conn.getOutputStream());
> 
>    Servlet servlet = getServletForRequest(request);
>    if(null == servlet)
>        servlet = defaultServlet;
> 
>    servlet.service(request, response);
> 
>    request.reset();
>    response.reset();
> }
> 
> In "real" Tomcat, each Connection object holds its own Request and
> Response objects ad manages them in a similar way, and of course,
> Tomcat can accept multiple simultaneous connections -- including
> multiple requests over a single connection -- simultaneously -- in the
> case of HTTP/2.
> 
> If you enable the RECYCLE_FACADES in Tomcat, the code changes to
> behave like this:
> 
> Connection conn = null;
> 
> while(null != (conn = socket.acceptConnection()) {
>    HttpServletRequest request = new TomcatHttpServletRequest();
>    HttpServletResponse response = new TomcatHttpServletResponse();
>    request.setRequestLine(conn.getRequestLine());
>    request.setInputStream(conn.getInputStream());
>    response.setOutputStream(conn.getOutputStream());
> 
>    Servlet servlet = getServletForRequest(request);
>    if(null == servlet)
>        servlet = defaultServlet;
> 
>    servlet.service(request, response);
> 
>    request.dispose();
>    response.dispose();
> }
> 
> Note how the request and response objects are no longer re-used across
> requests. This represents a trade-off between security/stability
> (always getting a fresh object) versus performance (less
> garbage-collection for a given request). An application can do things
> to the request or response that can break the way the server works, so
> an untrusted application should always be run with RECYCLE_FACADES set
> to ON.
> 
> If the application keeps a reference to a request or response object
> after the request has completed (as defined by returning from
> servlet.service()), then Bad Things can happen. If you write to that
> response's OutputStream, for example, you might write into the
> response of *another request* that is being handled after your code
> should be finished.
> 
>>>>
>>>> p.s. Speaking of the possibility of an improper implementation,
>>>> I've found some discussions which seem to have some similarity to
>>>> my case like the following:
>>>>
>>>> *
>>>> http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2
> 054107.html
>>>> : I've seen some similar Exceptions at the "MimeHeaders" class
>>>> which were recorded in my Tomcat's log too
>>>>
>>>> *
>>>> http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons
> es-td5034710.html
>>>> : A case where a bug in a Filter make Tomcat produce an invalid
>>>> HTTP response
>>>>
>>>> Those discussions made me think of this possibility where my app
>>>> might be improperly implemented. I'm going to check the application
>>>> code from this perspective on Monday.
> 
> When you use RECYCLE_FACADES, your application will only retain a
> reference to a useless object.... one which will likely NPE at some
> point when your code calls it. This makes the application safer
> (because one misbehaving servlet won't damage requests/responses
> currently in-use by subsequent requests) and also helps you find the
> error in your own code (because the problem will usually manifest
> itself as an earlier NPE instead of some "weirdness" which cannot
> really be readily explained because of the complexities of how objects
> are re-used, etc.).
> 
> Hope that helps,
> -chris
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org <ma...@tomcat.apache.org>
>> For additional commands, e-mail: users-help@tomcat.apache.org <ma...@tomcat.apache.org>
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat occasionally returns a response without HTTP headers

Posted by Kohei Nozaki <ko...@nozaki.me>.
Hello,

It turned out that the "sjsxp" library which JAX-WS RI v2.1.3 uses makes Tomcat behave this way. I tried a different version of JAX-WS RI (v2.1.7) which doesn't use the "sjsxp" library anymore and it solved the issue.

A very similar issue posted on the Metro mailing list: http://metro.1045641.n5.nabble.com/JAX-WS-RI-2-1-5-returning-malformed-response-tp1063518.html

It's surprising that a bug of a framework which is built on the Servlet API can make such an issue happen, but anyway thank you very much everyone who helped me out.

Regards,
Kohei

> On Dec 4, 2018, at 3:39, Christopher Schultz <ch...@christopherschultz.net> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Kohei,
> 
> On 12/1/18 10:06, Kohei Nozaki wrote:
>> Hello Konstantin, thanks for sharing the valuable information.
>> 
>>> On Dec 1, 2018, at 19:00, Konstantin Kolinko
>>> <kn...@gmail.com> wrote:
>>> 
>>>> * Our downstream Nginx instance (The client of our Tomcat
>>>> instance) recorded the error "upstream sent no valid HTTP/1.0
>>>> header while reading response header from upstream" at that
>>>> time and the error makes perfect sense concerning the response
>>>> which has neither HTTP status line nor HTTP headers.
>>> 
>>> 1. See the official FAQ / Troubleshoting page: 
>>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics
>>> 
>>> 
>>> 
> Especially pay attention to
>>> 1) configuring an access log 2) setting system property 
>>> org.apache.catalina.connector.RECYCLE_FACADES=true
>> 
>> I've investigated "RECYCLE_FACADES" and understand that an
>> improperly implemented application which keeps a reference to
>> Request or Response objects outside of their lifecycle can make
>> such an issue like mine happen (please correct me if I'm wrong..).
>> 
>> 
>> But I still don't quite understand what does "RECYCLE_FACADES=true"
>> do. The Wiki page says "This makes it easier to spot illegal access
>> when it happens, instead of waiting until side effects of such
>> access become visible" but how does it make easier? Does this
>> property make Tomcat produce an Exception or make Tomcat produce
>> some warning message to a log file or something when such accesses
>> happen, for example?
> 
> Tomcat usually handles requests something like this. Imagine a
> single-threaded server where Tomcat only accepts a single connection
> at a time (just to simplify the code to the point where it fits into a
> ML post).
> 
> Many of these methods are made-up. There is no
> TomcatHttpServletRequest class or a .setRequestLine method in it
> (though there are ... siilar concepts in there, way down deep). The
> point is how the objects are used, or rather *re* used.
> 
> HttpServletRequest request = new TomcatHttpServletRequest();
> HttpServletResponse response = new TomcatHttpServletResponse();
> 
> Connection conn = null;
> 
> while(null != (conn = socket.acceptConnection()) {
>    request.setRequestLine(conn.getRequestLine());
>    request.setInputStream(conn.getInputStream());
>    response.setOutputStream(conn.getOutputStream());
> 
>    Servlet servlet = getServletForRequest(request);
>    if(null == servlet)
>        servlet = defaultServlet;
> 
>    servlet.service(request, response);
> 
>    request.reset();
>    response.reset();
> }
> 
> In "real" Tomcat, each Connection object holds its own Request and
> Response objects ad manages them in a similar way, and of course,
> Tomcat can accept multiple simultaneous connections -- including
> multiple requests over a single connection -- simultaneously -- in the
> case of HTTP/2.
> 
> If you enable the RECYCLE_FACADES in Tomcat, the code changes to
> behave like this:
> 
> Connection conn = null;
> 
> while(null != (conn = socket.acceptConnection()) {
>    HttpServletRequest request = new TomcatHttpServletRequest();
>    HttpServletResponse response = new TomcatHttpServletResponse();
>    request.setRequestLine(conn.getRequestLine());
>    request.setInputStream(conn.getInputStream());
>    response.setOutputStream(conn.getOutputStream());
> 
>    Servlet servlet = getServletForRequest(request);
>    if(null == servlet)
>        servlet = defaultServlet;
> 
>    servlet.service(request, response);
> 
>    request.dispose();
>    response.dispose();
> }
> 
> Note how the request and response objects are no longer re-used across
> requests. This represents a trade-off between security/stability
> (always getting a fresh object) versus performance (less
> garbage-collection for a given request). An application can do things
> to the request or response that can break the way the server works, so
> an untrusted application should always be run with RECYCLE_FACADES set
> to ON.
> 
> If the application keeps a reference to a request or response object
> after the request has completed (as defined by returning from
> servlet.service()), then Bad Things can happen. If you write to that
> response's OutputStream, for example, you might write into the
> response of *another request* that is being handled after your code
> should be finished.
> 
>> 
>> p.s. Speaking of the possibility of an improper implementation,
>> I've found some discussions which seem to have some similarity to
>> my case like the following:
>> 
>> *
>> http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2
> 054107.html
>> : I've seen some similar Exceptions at the "MimeHeaders" class
>> which were recorded in my Tomcat's log too
>> 
>> *
>> http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons
> es-td5034710.html
>> : A case where a bug in a Filter make Tomcat produce an invalid
>> HTTP response
>> 
>> Those discussions made me think of this possibility where my app
>> might be improperly implemented. I'm going to check the application
>> code from this perspective on Monday.
> 
> When you use RECYCLE_FACADES, your application will only retain a
> reference to a useless object.... one which will likely NPE at some
> point when your code calls it. This makes the application safer
> (because one misbehaving servlet won't damage requests/responses
> currently in-use by subsequent requests) and also helps you find the
> error in your own code (because the problem will usually manifest
> itself as an earlier NPE instead of some "weirdness" which cannot
> really be readily explained because of the complexities of how objects
> are re-used, etc.).
> 
> Hope that helps,
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ <https://www.enigmail.net/>
> 
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwFeG4ACgkQHPApP6U8
> pFgxWxAAmG1UgpXnQUf2WdI6KQrgMa+Xx6nCkrr5XU/QK2EHvGO1jAE6Oc/qBmh4
> ak25KzQJbC25BDCar/JdFuUlxAVwt2qht7M3RoCVqKm/BoZuamnnVXO5fq8a15cW
> NWMXFm9Unmh2wJGI12U8ZuuaPwte0rs4MSn+84gIzI81iaPmQB2TD6ohrl4mtJAT
> LsZQQks0IJKpu6/kpx2z0DPcqRTIynRIeegBzkvUjXOHFzu5wF99ZnS2etsTEbr6
> bqKmEiSHSp9OjfiwggWMiHmnIPuZNwXziNj8cPpMxTWGLwMNXP1LUe7JSQ+UTKj4
> tF5HYZovYW95h9vRHV/SkiJULmzi2nR/TGCe6noz0cNYon3y7kQJdG3i4GPaQ+bs
> +aww7homCjIceWREQbUKIp+/WUNXmtN8f8EwRjq94Av73CTi5inT+gv377D0Y6fG
> ui/p2vxkENsRDn/E6khTywxYjxOTruT0cmeYJ4Ee45k4EqMTb7142fxwblq1sgUy
> Of3ziwYVW7i6gz5zJaxv6oHxYCKVlHwj8aaqaETon5clGE7yFrRyAsilsLhu86OO
> 0ZwFLGUw7JSsiNs/KPODhmsY7XhNevmUu7ejhyfPe/TMle+TQyUmjTrdnjc+yX3C
> NwNUnZQUoQhkCseMaC2tp+6I1fo4/iFBLYdfCTrZpffQy4L0Okg=
> =uLmM
> -----END PGP SIGNATURE-----
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org <ma...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org <ma...@tomcat.apache.org>

Re: Tomcat occasionally returns a response without HTTP headers

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Kohei,

On 12/1/18 10:06, Kohei Nozaki wrote:
> Hello Konstantin, thanks for sharing the valuable information.
> 
>> On Dec 1, 2018, at 19:00, Konstantin Kolinko
>> <kn...@gmail.com> wrote:
>> 
>>> * Our downstream Nginx instance (The client of our Tomcat
>>> instance) recorded the error "upstream sent no valid HTTP/1.0
>>> header while reading response header from upstream" at that
>>> time and the error makes perfect sense concerning the response
>>> which has neither HTTP status line nor HTTP headers.
>> 
>> 1. See the official FAQ / Troubleshoting page: 
>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics
>>
>>
>> 
Especially pay attention to
>> 1) configuring an access log 2) setting system property 
>> org.apache.catalina.connector.RECYCLE_FACADES=true
> 
> I've investigated "RECYCLE_FACADES" and understand that an
> improperly implemented application which keeps a reference to
> Request or Response objects outside of their lifecycle can make
> such an issue like mine happen (please correct me if I'm wrong..).
> 
> 
> But I still don't quite understand what does "RECYCLE_FACADES=true"
> do. The Wiki page says "This makes it easier to spot illegal access
> when it happens, instead of waiting until side effects of such
> access become visible" but how does it make easier? Does this
> property make Tomcat produce an Exception or make Tomcat produce
> some warning message to a log file or something when such accesses
> happen, for example?

Tomcat usually handles requests something like this. Imagine a
single-threaded server where Tomcat only accepts a single connection
at a time (just to simplify the code to the point where it fits into a
ML post).

Many of these methods are made-up. There is no
TomcatHttpServletRequest class or a .setRequestLine method in it
(though there are ... siilar concepts in there, way down deep). The
point is how the objects are used, or rather *re* used.

HttpServletRequest request = new TomcatHttpServletRequest();
HttpServletResponse response = new TomcatHttpServletResponse();

Connection conn = null;

while(null != (conn = socket.acceptConnection()) {
    request.setRequestLine(conn.getRequestLine());
    request.setInputStream(conn.getInputStream());
    response.setOutputStream(conn.getOutputStream());

    Servlet servlet = getServletForRequest(request);
    if(null == servlet)
        servlet = defaultServlet;

    servlet.service(request, response);

    request.reset();
    response.reset();
}

In "real" Tomcat, each Connection object holds its own Request and
Response objects ad manages them in a similar way, and of course,
Tomcat can accept multiple simultaneous connections -- including
multiple requests over a single connection -- simultaneously -- in the
case of HTTP/2.

If you enable the RECYCLE_FACADES in Tomcat, the code changes to
behave like this:

Connection conn = null;

while(null != (conn = socket.acceptConnection()) {
    HttpServletRequest request = new TomcatHttpServletRequest();
    HttpServletResponse response = new TomcatHttpServletResponse();
    request.setRequestLine(conn.getRequestLine());
    request.setInputStream(conn.getInputStream());
    response.setOutputStream(conn.getOutputStream());

    Servlet servlet = getServletForRequest(request);
    if(null == servlet)
        servlet = defaultServlet;

    servlet.service(request, response);

    request.dispose();
    response.dispose();
}

Note how the request and response objects are no longer re-used across
requests. This represents a trade-off between security/stability
(always getting a fresh object) versus performance (less
garbage-collection for a given request). An application can do things
to the request or response that can break the way the server works, so
an untrusted application should always be run with RECYCLE_FACADES set
to ON.

If the application keeps a reference to a request or response object
after the request has completed (as defined by returning from
servlet.service()), then Bad Things can happen. If you write to that
response's OutputStream, for example, you might write into the
response of *another request* that is being handled after your code
should be finished.

> 
> p.s. Speaking of the possibility of an improper implementation,
> I've found some discussions which seem to have some similarity to
> my case like the following:
> 
> *
> http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2
054107.html
> : I've seen some similar Exceptions at the "MimeHeaders" class
> which were recorded in my Tomcat's log too
> 
> *
> http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons
es-td5034710.html
> : A case where a bug in a Filter make Tomcat produce an invalid
> HTTP response
> 
> Those discussions made me think of this possibility where my app
> might be improperly implemented. I'm going to check the application
> code from this perspective on Monday.

When you use RECYCLE_FACADES, your application will only retain a
reference to a useless object.... one which will likely NPE at some
point when your code calls it. This makes the application safer
(because one misbehaving servlet won't damage requests/responses
currently in-use by subsequent requests) and also helps you find the
error in your own code (because the problem will usually manifest
itself as an earlier NPE instead of some "weirdness" which cannot
really be readily explained because of the complexities of how objects
are re-used, etc.).

Hope that helps,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwFeG4ACgkQHPApP6U8
pFgxWxAAmG1UgpXnQUf2WdI6KQrgMa+Xx6nCkrr5XU/QK2EHvGO1jAE6Oc/qBmh4
ak25KzQJbC25BDCar/JdFuUlxAVwt2qht7M3RoCVqKm/BoZuamnnVXO5fq8a15cW
NWMXFm9Unmh2wJGI12U8ZuuaPwte0rs4MSn+84gIzI81iaPmQB2TD6ohrl4mtJAT
LsZQQks0IJKpu6/kpx2z0DPcqRTIynRIeegBzkvUjXOHFzu5wF99ZnS2etsTEbr6
bqKmEiSHSp9OjfiwggWMiHmnIPuZNwXziNj8cPpMxTWGLwMNXP1LUe7JSQ+UTKj4
tF5HYZovYW95h9vRHV/SkiJULmzi2nR/TGCe6noz0cNYon3y7kQJdG3i4GPaQ+bs
+aww7homCjIceWREQbUKIp+/WUNXmtN8f8EwRjq94Av73CTi5inT+gv377D0Y6fG
ui/p2vxkENsRDn/E6khTywxYjxOTruT0cmeYJ4Ee45k4EqMTb7142fxwblq1sgUy
Of3ziwYVW7i6gz5zJaxv6oHxYCKVlHwj8aaqaETon5clGE7yFrRyAsilsLhu86OO
0ZwFLGUw7JSsiNs/KPODhmsY7XhNevmUu7ejhyfPe/TMle+TQyUmjTrdnjc+yX3C
NwNUnZQUoQhkCseMaC2tp+6I1fo4/iFBLYdfCTrZpffQy4L0Okg=
=uLmM
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat occasionally returns a response without HTTP headers

Posted by Kohei Nozaki <ko...@nozaki.me>.
Hello Konstantin, thanks for sharing the valuable information.

> On Dec 1, 2018, at 19:00, Konstantin Kolinko <kn...@gmail.com> wrote:
> 
>> * Our downstream Nginx instance (The client of our Tomcat instance) recorded the error "upstream sent no valid HTTP/1.0 header while reading response header from upstream" at that time and the error makes perfect sense concerning the response which has neither HTTP status line nor HTTP headers.
> 
> 1. See the official FAQ / Troubleshoting page:
> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics
> 
> Especially pay attention to
> 1) configuring an access log
> 2) setting system property
> org.apache.catalina.connector.RECYCLE_FACADES=true

I've investigated "RECYCLE_FACADES" and understand that an improperly implemented application which keeps a reference to Request or Response objects outside of their lifecycle can make such an issue like mine happen (please correct me if I'm wrong..). 

But I still don't quite understand what does "RECYCLE_FACADES=true" do. The Wiki page says "This makes it easier to spot illegal access when it happens, instead of waiting until side effects of such access become visible" but how does it make easier? Does this property make Tomcat produce an Exception or make Tomcat produce some warning message to a log file or something when such accesses happen, for example?

p.s. Speaking of the possibility of an improper implementation, I've found some discussions which seem to have some similarity to my case like the following:

* http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2054107.html : I've seen some similar Exceptions at the "MimeHeaders" class which were recorded in my Tomcat's log too

* http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-responses-td5034710.html : A case where a bug in a Filter make Tomcat produce an invalid HTTP response

Those discussions made me think of this possibility where my app might be improperly implemented. I'm going to check the application code from this perspective on Monday..

Best Regards,
Kohei


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org