Safely restart servers?

[Phillip B Oldham <ph...@gmail.com>]

How would I safely restart a server (python, in this case, but in
generally any language), so that it completes any current requests
before exiting?

-- 
Phillip B Oldham
phillip.oldham@gmail.com
+44 (0) 7525 01 09 01

RE: Safely restart servers?

[James King <Ja...@Dell.com>]

Each runtime is different.  I looked over the python TServer.  It looks like it will process a client forever until the client disconnects.  There doesn't appear to be any mechanism for stopping the TServer in python.  My guess is that you close your instance of the TServer.serverTransport to stop listening, and then your serve() thread, or in the case of multithread each handling thread, will continue to run until the client disconnects due to the tight while loop in the serve() routine.  This code doesn't check any "stop" flag which would allow it to bail out on the next request in python.  So my short answer is that it looks like it would complete any outstanding requests, and in fact any additional requests until the client disconnects even after you close the listening socket.  If that's not the case, please set me straight python users!

In the current codebase, the C# runtime servers behave the same way.  They will close the listening socket in response to a close() call, however the serving thread will continue to process requests from the client until it disconnects because of the (while process ...) loop.  In the patch for THRIFT-66 I fixed this on the legacy TServer classes so that the next request from the client will be processed and then the server will stop serving because it now checks the stop flag in that while loop.  The multithreaded servers in C# are similar.

In the patch for THRIFT-66 I introduced Endpoints and Channels in C#.  Endpoints allow for two-way communication and multiple outstanding requests; channels allow for multiple thrift services on a single connection.  Endpoints handle Stop() as an immediate disconnect.  When a TListeningEndpoint is closed, it closes all the associated TConnectedEndpoints (one for each client connected).  When each TConnectedEndpoint is closed, the socket is closed immediately.  Requests currently being processed will be processed fully, since each request is handled on a separate thread (you can have multiple outstanding requests on a connection with endpoints), and endpoints buffer the responses to serialize delivery of the response back to the socket (which has closed: the client will never receive the response, but the request was fully processed).  This could be changed so that on an endpoint close, the socket shuts down the inbound; processes all outstanding requests; then shuts down the outbound as well, finally closing the socket.  I suspect the transport layers would need to change a little to properly support the socket shutdown(0, 1, 2) notion though.

James E. King, III                       
Senior Software Engineer                 
Dell (EqualLogic) HIT Team (ASM)      
300 Innovative Way, Suite 301   
Nashua, NH 03062

-----Original Message-----
From: Phillip B Oldham [mailto:phillip.oldham@gmail.com] 
Sent: Wednesday, August 18, 2010 5:44 AM
To: thrift-user
Subject: Safely restart servers?

How would I safely restart a server (python, in this case, but in generally any language), so that it completes any current requests before exiting?

--
Phillip B Oldham
phillip.oldham@gmail.com
+44 (0) 7525 01 09 01