Filter and ServletResponseWrapper obligations

[Michael Ludwig <mi...@gmx.de>]

This is both Tomcat 5.5.27 and 6.0.18. (And also Jetty 6.1.18.)

Consider the following servlet code:

  PrintWriter out = res.getWriter();
  ...
  out.println( "<p>before include</p>");
  rd = context.getRequestDispatcher( "/include.html");
  // contains <p>included HTML</p>
  rd.include( req, res);
  out.println( "<p>after include</p>");
  out.flush();
  out.close();

This displays alright.

  before include
  included HTML
  after include

But then, this is without any filters. Let's add a filter.

  included HTML
  before include
  after include

This is wrong. The output order is mangled. What happened?

In this case the included file "/include.html" is processed by Tomcat's
DefaultServlet. Let's try the same code with a JSP, "/include.jsp".

  before include
  included JSP
  after include

This works correctly, with or without the filter. Why does it work
with a JSP and not with an HTML page? Obviously, these are processed
differently. But how exactly?

I can make the included HTML page display correctly even in the face
of my filter by adding the following statement directly before the
include();

  out.flush();

But is it the servlet's obligation to flush() the buffer before doing an
include? The documentation for RequestDispatcher doesn't say anything to
that effect.

I can also make the included HTML page display correctly despite the
filter by resorting to res.getOutputStream() instead of res.getWriter()
in my servlet code. But it should work either way, shouldn't it?

Let's scroll back a little. What filter did I set up that provoked
this odd behaviour? The filter is trivial, I don't think it matters.
The decisive issue seems to be what's going on in the subclass of
HttpServletResponseWrapper which is a helper to the filter.

 public ServletOutputStream getOutputStream() throws IOException {
  // getResponse().getOutputStream();
  return this.stream;
 }

 public PrintWriter getWriter() throws IOException {
  // getResponse().getWriter();
  return this.writer;
 }

You see there are calls to the underlying response object, which are
commented out. When I comment these lines back in, everything works
fine. No need to flush the buffer before the include, to resort to SOS
instead of PW, or to JSP instead of HTML.

So these calls on the underlying object seem to have something to do
with the behaviour observed. Can anyone explain what's going on?

Note the problem is manifest only for all of the following conditions:

* include() of a static HTML file via DefaultServlet;
* output done using the PrintWriter;
* no flush() on the buffer prior to the include();
* addition of a filter lacking calls to the underlying response
  object in getOutputStream() and getWriter().

Another way of putting this is probably: What is the contract the
Filter and the ServletResponseWrapper have to fulfill in order to be
well-behaved Tomcat citizens? Or, for that matter, web container
citizens? What are their obligations?

Is this contract documented? Alright, I found the spec.

http://jcp.org/aboutJava/communityprocess/mrel/jsr154/index2.html

But I haven't read it yet :-) So if anyone wants to comment ...

Thanks,

Michael Ludwig

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Filter and ServletResponseWrapper obligations

[Michael Ludwig <mi...@gmx.de>]

Christopher Schultz schrieb am 01.12.2008 um 14:15:31 (-0500):
> Michael Ludwig wrote:

Hi Chris,

thanks for your reply. Sorry for not getting back earlier.

> Is this the filter you wrote in your other thread?

Yes.

> Maybe you are not printing your output at the right time. Or maybe you
> aren't flushing buffers at the right time.

See below.

> > I can also make the included HTML page display correctly despite
> > the filter by resorting to res.getOutputStream() instead of
> > res.getWriter() in my servlet code. But it should work either way,
> > shouldn't it?
> 
> That is definitely odd.

The source code of DefaultServlet, which serves static HTML, explains
this behaviour.

> >  public ServletOutputStream getOutputStream() throws IOException {
> >   // getResponse().getOutputStream();
> >   return this.stream;
> >  }
> > 
> >  public PrintWriter getWriter() throws IOException {
> >   // getResponse().getWriter();
> >   return this.writer;
> >  }
> > 
> > You see there are calls to the underlying response object, which are
> > commented out. When I comment these lines back in, everything works
> > fine. No need to flush the buffer before the include, to resort to
> > SOS instead of PW, or to JSP instead of HTML.
> 
> Then you should definitely do this ;)

Yes, these calls are needed.

> > So these calls on the underlying object seem to have something to do
> > with the behaviour observed. Can anyone explain what's going on?
> 
> I'll bet that since the response hasn't had the writer/outputstream
> choice made, the DefaultServlet makes a random decision. If your
> wrapper response has made a different choice, things can get fouled
> up.

This is the relevant chunk from serveResource() in DefaultServlet (which
serves static HTML, among other things):

	try {
		ostream = response.getOutputStream();
	} catch (IllegalStateException e) {
		// If it fails, we try to get a Writer instead if we're
		// trying to serve a text file
		if ( (contentType == null)
				|| (contentType.startsWith("text"))
				|| (contentType.endsWith("xml")) ) {
			writer = response.getWriter();
		} else {
			throw e;
		}
	}

A filter intercepting the response and deviating it in its own buffers
has to make sure the call to getOutputStream() fails when it has already
issued a PrintWriter.

Michael Ludwig

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Filter and ServletResponseWrapper obligations

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael,

Michael Ludwig wrote:
> But then, this is without any filters. Let's add a filter.
> 
>   included HTML
>   before include
>   after include
> 
> This is wrong. The output order is mangled. What happened?

Is this the filter you wrote in your other thread? Maybe you are not
printing your output at the right time. Or maybe you aren't flushing
buffers at the right time.

> I can make the included HTML page display correctly even in the face
> of my filter by adding the following statement directly before the
> include();
> 
>   out.flush();
> 
> But is it the servlet's obligation to flush() the buffer before doing an
> include? The documentation for RequestDispatcher doesn't say anything to
> that effect.

Ideally, the buffer itself is always in order, and flushing the buffer
merely pushes everything out of the buffer back to the client. Since you
are doing some of your own buffering, you need to make sure that
everything is sane in that buffer.

> I can also make the included HTML page display correctly despite the
> filter by resorting to res.getOutputStream() instead of res.getWriter()
> in my servlet code. But it should work either way, shouldn't it?

That is definitely odd.

> Let's scroll back a little. What filter did I set up that provoked
> this odd behaviour? The filter is trivial, I don't think it matters.
> The decisive issue seems to be what's going on in the subclass of
> HttpServletResponseWrapper which is a helper to the filter.
> 
>  public ServletOutputStream getOutputStream() throws IOException {
>   // getResponse().getOutputStream();
>   return this.stream;
>  }
> 
>  public PrintWriter getWriter() throws IOException {
>   // getResponse().getWriter();
>   return this.writer;
>  }
> 
> You see there are calls to the underlying response object, which are
> commented out. When I comment these lines back in, everything works
> fine. No need to flush the buffer before the include, to resort to SOS
> instead of PW, or to JSP instead of HTML.

Then you should definitely do this ;)

> So these calls on the underlying object seem to have something to do
> with the behaviour observed. Can anyone explain what's going on?

I'll bet that since the response hasn't had the writer/outputstream
choice made, the DefaultServlet makes a random decision. If your wrapper
response has made a different choice, things can get fouled up.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkk0N9MACgkQ9CaO5/Lv0PBPmgCfWNV2iez1hpXOra+GdlIRRPPe
pnYAnA2f+dmBcmuk42sxt2n/4bbfQIvb
=5Fi1
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org