RE: Possible IIS SPDY Redirector for Tomcat

[Konstantin Preißer <kp...@apache.org>]

Hi all,

I would like to let you know some updates on this:

1) Results of performance measurement with various configurations and the SPDY-Redirector
2) Switching to Async I/O for the implementation of the IIS module


1) Performance

> -----Original Message-----
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Sent: Monday, September 30, 2013 8:53 PM
> To: Tomcat Developers List
> Subject: Re: Possible IIS SPDY Redirector for Tomcat
> 
> Konstantin,
> 
> > So it seems that the main performance problem is IIS when using
> > managed code/ASP.Net to write to the response, but I need to do
> > additional testing.
> 
> Wow. Are you sure the thread burning the CPU is the one running the
> above code? Seems ... unfortunate.

I have now done additional testing with various configurations and I think my previous statement was wrong - IIS is not the culprit here. I guess it's the multiplexing that the SPDY Redirector is doing (multiplexing multiple HTTP requests on a single TCP connection) that causes the CPU usage. However, I don't think that there is a real problem here comparing the performance/CPU usage to Jetty.

To measure performance (speed and CPU usage), I set up a 2-CPU VM with IIS 8.5 (Windows Server 2012 R2 x64), Tomcat 7.0.42 (for ISAPI Redirector) and Jetty 8.1.13 (for SPDY Redirector). For IIS, I created one web application using the SPDY Redirector (connecting to Jetty) and one web application using the ISAPI Redirector (connecting to Tomcat).
I then created a Servlet 3.0 webapp with a Servlet that produces ~ 700 MB of random data, sending it in 32 KB chunks to the client, using the following code:

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.setContentType("text/plain");
        response.setCharacterEncoding("UTF-8");
        response.addHeader("Content-Disposition", "attachment; filename=\"myTextfile.txt\"");

        byte[] bytes = new byte[32768];
        for (int i = 0; i < bytes.length; i++) {
            bytes[i] = (byte)'a';
        }
        
        long bytesWritten = 0;
        long lastDisplay = 0;
        try (OutputStream out = response.getOutputStream()) {
            while (true) {
                out.write(bytes);
                bytesWritten += bytes.length;
                
                if (bytesWritten - lastDisplay > 500 * 1024) {
                    lastDisplay = bytesWritten + 500 * 1024;
                    System.out.println(bytesWritten + " Bytes written (completely).");
                }
                
                if (bytesWritten > 700 * 1024 * 1024)
                    break;
            }
        }
    }


Then, I created a C# application that creates a TCP connection and sending the following raw request:
GET /TestSpdyServer/BigFileProducerServlet HTTP/1.1
Host: localhost
Connection: close

and reads from the TCP connection until it is closed, counting the bytes and milliseconds to calculate the average speed.
I then tested the speed when 1) connecting directly to Jetty (HTTP), 2) connecting directly to Tomcat (HTTP), 3)+4) connecting to IIS with SPDY-Redirector, 5) connecting to IIS with ISAPI redirector, 6) connecting to IIS using a ASP.Net handler producing 700 MB of random data, equally to the Servlet above.
This is what I got:

1) JettyHttp-Direct: 260 MB/s (Jetty: 47% CPU)
2) TomcatHttpBio-Direct: 275 MB/s (Tomcat: 45% CPU)
3) Iis-SpdyRedirector(Async)-JettySpdy (64 KB initial window size): 170 MB/s (Jetty: 35% CPU, IIS: 43% CPU)
4) Iis-SpdyRedirector(Async)-JettySpdy (128 KB initial window size): 205 MB/s (Jetty: 40% CPU, IIS: 50% CPU)
5) Iis-IsapiRedirector-TomcatAjpBio: 190 MB/s (Tomcat: 10% CPU, IIS: 20% CPU)
6) Iis-Asp.Net-Handler-Direct: 400 MB/s, 50% CPU

As you can see, with the SPDY redirector, both Jetty and IIS have a high amount of CPU usage (probably the costs of multiplexing) whereas the ISAPI redirector had a low CPU usage. However, I'm pretty happy with the current performance of the SPDY redirector.


2) Switching the implementation of the IIS module to Async I/O - see [1]: "Scalable Apps with Asynchronous Programming in ASP.NET"

.Net 4.5 introduces new "async" and "await" keywords for C# that allow you to easily create asynchronous code. E.g. you can write code just as you would with synchronous operations, and the compiler will create the Async State Machine for you.

As the previous implementation of the SPDY Redirector used blocking/synchronous IO for the IIS module, it meant that 1 thread was used per request. Now imagine you have 200 clients that simultaneously do requests to a resource that produces the response body very slowly, it would mean that IIS had to create 200 threads for serving the requests. This can be a problem also e.g. for long-polling comet applications (where it may take some time until the HTTP response is sent) or Websocket connections (however WebSockets with IIS 8 can only be used with async IO so I had to do the switch to Async anyway for being able to support Websocket).

For example, with Servlet 3.1 you can use non-blocking I/O so that you don't need to acquire a thread for the complete duration of a request.

Therefore, I now switched the IIS module implementation of the SPDY redirector to async IO (with the "async" and "await" keywords this was rather simple). The result is that if a request takes a long time to complete, IIS does not need to acquire a thread for the complete duration of the request - instead it uses a thread from its Threadpool only if there is new data to be forwarded. (The underlying SPDY connector still uses blocking I/O for the SPDY connection, but since one SPDY connection is intended to multiplex a huge amount of HTTP requests, atm there is no need for the SPDY connector to use async I/O for the SPDY TCP connection).


For example, the ISAPI redirector seems to use blocking I/O when forwarding HTTP requests to Tomcat. I tested this by creating a servlet that writes 100 lines of text and waiting 10 seconds after writing each line. Then, I made a C# application creating 170 TCP connections and simultaneously sending a request on each connection.

Then I monitored the number of threads of the IIS worker processes (w3wp.exe) before and after sending the requests:
IIS with SPDY Redirector: 29 Threads -> 35 Threads
IIS with ISAPI Redirector: 26 Threads -> 195 Threads (the additional threads are also created very slowly - I saw 1 thread per second - which means there is additional waiting time for the clients)

As you can see, with the async implementation of the IIS module of the SPDY redirector the number of threads did not significantly increase when doing 170 concurrent requests (initially I wanted to try more, but Jetty does not allow this - after about 180 requests are created on the SPDY connection, Jetty simply sends a GOAWAY frame and closes the TCP connection, effectively aborting all running HTTP requests).

Also, as AJP needs one TCP connection for each request, doing 170 concurrent requests means the ISAPI redirector has to establish 170 separate TCP connections to Tomcat, whereas the SPDY redirector only needs a single TCP connection where it can multiplex the concurrent requests.


The SPDY Redirector does not yet support redirecting WebSocket connections, but I think I will look into the "WebSocket Layering over SPDY/3" [1] document (and the Websocket protocol spec) to implement redirecting Websocket connections (Jetty 8.1 seem to support Websockets while it is only Servlet 3.0, whereas Jetty 9.1 should support Servlet 3.1 but I was not yet able to create a non-SSL SPDY/3 connector with it).


If you would like to test the SPDY redirector with a downloadable version of IIS 8 (that runs also on client Windows editions), you can find the instructions for setting it up at [2]. I would appreciate if someone could test the SPDY redirector for some feedback.


What do you think?


Thanks!

Regards,
Konstantin Preißer


[1] http://msdn.microsoft.com/en-us/magazine/cc163463.aspx
[2] http://markmail.org/message/atwky5b4qpm52qq3


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org