Asynchronous request processing

[Rey Francois <fr...@capco.com>]

We are currently investigating the possibility to service HTTP requests with
an asynchronous model. The idea is to retrieve the dynamic content from the
back-end using asynchronous calls. The motivations are: 
*	Sometimes the existing backend systems already have an asynchronous
interface 
*	The increased response time when building a page which dynamic
content is retrieved from multiple calls to the backend: these calls can be
executed concurrently. 
*	Better scalability: the web server does not need to wait idly for
the results
The difficulty to implement such a mechanism is due to the Servlet spec
itself: once the service() method returns, container can assume that the
servicing of the request is finished and the response object should be
closed. This means that while waiting for the results of the operations in
the backend, we cannot just simply return the control to the container.
Rather then having the servicing thread actively wait for the results, we're
thinking about putting it to sleep and awake it when the results are ready.
One way to achieve this is the use of the Object.wait() and Object.notify()
methods. The code in the ActionForm.perform() would look more or less like
this:
	Action.perform() {
			ResultHandler resultHandler = new
ResultHandler(request, response) {
				handleResult(command) {
			  			// process results
						// when all results have
arrived, call notify()
				}
			};
			Command c1 = Command.getCommand("COMMAND1");
			c1.execute(resultHandler); // async. call
			Command c2 = Command.getCommand("COMMAND2");
			c.execute(resultHandler); // async call
			Try {
			resultHandler.acceptResults(long timeout) ;
			} catch (TimeOutException) {
				// time out error handling
			}
			// from here all results have been returned.
			...
			return new ActionForward(...) ;
	}
The ResultHandler class would look somehow like this:

	public class ResultHandler {
	synchronized void acceptResults(long timeout) throws
TimeOutException {
			wait(timeout);
			if (!areAllCommandFinished) {
				throw new TimeOutException(...);
			}
	}
	// Method called internally in order to register the commands
	// this handler is waiting for
	void registerCommand(Command command) {...}

	// Return true if all registered commands have terminated
	boolean areAllCommandFinished() {...}

	// Method called internally when results arrive
	void internalHandleResults(Command command) {
		// Check if the command is registered and mark it as
terminated
		// then call the user's logic for processing results
		handleResults(command);
		// If all commands are finished, awake servicing thread
		if (areAllCommandFinished) {
			synchronize(this) {
				notify();
			}
		}
	}

	// Method to be overridden by the user 
	abstract synchronized void handleResults(Command command);
	}
The code above is just indicative and may contains syntax or design errors,
but I think the idea is well illustrated.
Does anybody have any comments on this approach? I'm also particularly
interested in your comments regarding the fact that the servicing thread is
put to "sleep": is there any risk in doing this? Can this have any conflict
with the container and the way it manages its threads?

Thanks for your feedback.

François Rey
Financial WebSuite
The Capital Markets Company
http://www.capco.com/


************************************************************************
The information in this email is confidential and is intended solely
for the addressee(s).
Access to this email by anyone else is unauthorised. If you are not
an intended recipient, you must not read, use or disseminate the
information contained in the email.
Any views expressed in this message are those of the individual sender,
except where the sender specifically states them to be the views of
The Capital Markets Company.

http://www.capco.com
***********************************************************************

Re: Asynchronous request processing

["Craig R. McClanahan" <Cr...@eng.sun.com>]

Rey Francois wrote:

> We are currently investigating the possibility to service HTTP requests with
> an asynchronous model. The idea is to retrieve the dynamic content from the
> back-end using asynchronous calls. The motivations are:
> *       Sometimes the existing backend systems already have an asynchronous
> interface
> *       The increased response time when building a page which dynamic
> content is retrieved from multiple calls to the backend: these calls can be
> executed concurrently.
> *       Better scalability: the web server does not need to wait idly for
> the results
> The difficulty to implement such a mechanism is due to the Servlet spec
> itself: once the service() method returns, container can assume that the
> servicing of the request is finished and the response object should be
> closed. This means that while waiting for the results of the operations in
> the backend, we cannot just simply return the control to the container.
> Rather then having the servicing thread actively wait for the results, we're
> thinking about putting it to sleep and awake it when the results are ready.
> One way to achieve this is the use of the Object.wait() and Object.notify()
> methods. The code in the ActionForm.perform() would look more or less like
> this:
>         Action.perform() {
>                         ResultHandler resultHandler = new
> ResultHandler(request, response) {
>                                 handleResult(command) {
>                                                 // process results
>                                                 // when all results have
> arrived, call notify()
>                                 }
>                         };
>                         Command c1 = Command.getCommand("COMMAND1");
>                         c1.execute(resultHandler); // async. call
>                         Command c2 = Command.getCommand("COMMAND2");
>                         c.execute(resultHandler); // async call
>                         Try {
>                         resultHandler.acceptResults(long timeout) ;
>                         } catch (TimeOutException) {
>                                 // time out error handling
>                         }
>                         // from here all results have been returned.
>                         ...
>                         return new ActionForward(...) ;
>         }
> The ResultHandler class would look somehow like this:
>
>         public class ResultHandler {
>         synchronized void acceptResults(long timeout) throws
> TimeOutException {
>                         wait(timeout);
>                         if (!areAllCommandFinished) {
>                                 throw new TimeOutException(...);
>                         }
>         }
>         // Method called internally in order to register the commands
>         // this handler is waiting for
>         void registerCommand(Command command) {...}
>
>         // Return true if all registered commands have terminated
>         boolean areAllCommandFinished() {...}
>
>         // Method called internally when results arrive
>         void internalHandleResults(Command command) {
>                 // Check if the command is registered and mark it as
> terminated
>                 // then call the user's logic for processing results
>                 handleResults(command);
>                 // If all commands are finished, awake servicing thread
>                 if (areAllCommandFinished) {
>                         synchronize(this) {
>                                 notify();
>                         }
>                 }
>         }
>
>         // Method to be overridden by the user
>         abstract synchronized void handleResults(Command command);
>         }
> The code above is just indicative and may contains syntax or design errors,
> but I think the idea is well illustrated.
> Does anybody have any comments on this approach? I'm also particularly
> interested in your comments regarding the fact that the servicing thread is
> put to "sleep": is there any risk in doing this? Can this have any conflict
> with the container and the way it manages its threads?
>

This approach looks like it should be safe -- from the perspective of the
servlet container, the call is still synchronous.  Where you get into trouble is
trying to reference the request and response objects of a particular request
after the service() method has returned, which does not happen in your scenario.

One caveat is that some servlet containers might run your web apps under a
SecurityManager that does not allow asynchronous threads at all, so you will
want to make sure that this is allowed.

An alternative approach to asynchronous processing is to remember that the user
might want to see some intermediate feedback while the asynchronous commands are
executing.  You might think about an Action designed (in general) like this:

    if (background-commands-not-started) {
        start-background-commands (in separate threads)
        return "Work in progress" page
    } else if (background-commands-not-finished) {
        return "Work still in progress" page
    } else {
        collect-final-results
        return "here is the answers" page
    }

and you could rig the work in progress pages to do a meta-refresh (back to the
same answer) every few seconds.  As long as you ensure that you've collected all
required parameters from the original request *before* returning, the
asynchronous command execution threads can run with no difficulty.

>
> Thanks for your feedback.
>
> François Rey
> Financial WebSuite
> The Capital Markets Company
> http://www.capco.com/
>

Craig McClanahan