You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by "Christian, Daniel" <Da...@Honeywell.com> on 2012/01/27 22:11:54 UTC

How to have a server manage sub-processes?

My server process needs to spawn some sub processes and interact with them occasionally over stdin/stdout (and notice if they die).

What's the  best way to do that with thrift?

My server is in python on Linux.  I'm using thrift 0.8.

Thanks,
-Dan


Re: How to have a server manage sub-processes?

Posted by Rush Manbert <ru...@manbert.com>.
Thrift is about two things:
* Easily seralizable data objects (there are many uses for that alone)
* RPC using those data objects

so its job is to let your clients talk to your server and vice versa. But how your server spawns and manages subprocesses or threads is not part of thrift. The threaded server  allows for a server thread per transaction, but once again if the server thread wants to spawn subthreads or subprocesses that is done independently of Thrift.

I think the short answer to your question "What's the best way to do that with Thrift?" is that you don't do that with Thrift.

I don't feel like I'm helping you particularly well here, it may be that I don't really understand your question.

- Rush

On Jan 30, 2012, at 7:46 AM, Christian, Daniel wrote:

> I didn't see any supported way to add my file descriptors to thrift, get thrift to (non-destructively) timeout from read, or export thrift file descriptors to an external select/poll (the latter seems possible, it just doesn't look "supported").
> 
> Am I missing something?  I'm new to thrift, so I'm still learning about it.
> 
> -Dan
> 
> -----Original Message-----
> From: Rush Manbert [mailto:rush@manbert.com] 
> Sent: Saturday, January 28, 2012 11:06 AM
> To: user@thrift.apache.org
> Subject: Re: How to have a server manage sub-processes?
> 
> Hi Dan,
> 
> Unless I have missed something since we froze the Thrift version we use, this really has nothing to do with Thrift. Thrift uses posix to implement its internal scheduler, but you are not limited by that at all. (You certainly don't need to use the Thrift scheduler.) We use boost threads in our code.
> 
> - Rush
> 
> On Jan 27, 2012, at 1:11 PM, Christian, Daniel wrote:
> 
>> My server process needs to spawn some sub processes and interact with them occasionally over stdin/stdout (and notice if they die).
>> 
>> What's the  best way to do that with thrift?
>> 
>> My server is in python on Linux.  I'm using thrift 0.8.
>> 
>> Thanks,
>> -Dan


RE: How to have a server manage sub-processes?

Posted by "Christian, Daniel" <Da...@Honeywell.com>.
I didn't see any supported way to add my file descriptors to thrift, get thrift to (non-destructively) timeout from read, or export thrift file descriptors to an external select/poll (the latter seems possible, it just doesn't look "supported").

Am I missing something?  I'm new to thrift, so I'm still learning about it.

-Dan

-----Original Message-----
From: Rush Manbert [mailto:rush@manbert.com] 
Sent: Saturday, January 28, 2012 11:06 AM
To: user@thrift.apache.org
Subject: Re: How to have a server manage sub-processes?

Hi Dan,

Unless I have missed something since we froze the Thrift version we use, this really has nothing to do with Thrift. Thrift uses posix to implement its internal scheduler, but you are not limited by that at all. (You certainly don't need to use the Thrift scheduler.) We use boost threads in our code.

- Rush

On Jan 27, 2012, at 1:11 PM, Christian, Daniel wrote:

> My server process needs to spawn some sub processes and interact with them occasionally over stdin/stdout (and notice if they die).
> 
> What's the  best way to do that with thrift?
> 
> My server is in python on Linux.  I'm using thrift 0.8.
> 
> Thanks,
> -Dan


Re: How to have a server manage sub-processes?

Posted by Rush Manbert <ru...@manbert.com>.
Hi Dan,

Unless I have missed something since we froze the Thrift version we use, this really has nothing to do with Thrift. Thrift uses posix to implement its internal scheduler, but you are not limited by that at all. (You certainly don't need to use the Thrift scheduler.) We use boost threads in our code.

- Rush

On Jan 27, 2012, at 1:11 PM, Christian, Daniel wrote:

> My server process needs to spawn some sub processes and interact with them occasionally over stdin/stdout (and notice if they die).
> 
> What's the  best way to do that with thrift?
> 
> My server is in python on Linux.  I'm using thrift 0.8.
> 
> Thanks,
> -Dan


Re: How to have a server manage sub-processes?

Posted by Dvir Volk <dv...@doat.com>.
makes sense. you might want to use the non blocking server in this
situation, to save on threads and allow for more concurrency, but the
principal remains the same.
in my case I've designed my app and its sub processes around thrift, so it
simplified things, apart from sharing state between them, which is an
independent problem.

On Mon, Feb 27, 2012 at 9:25 PM, Christian, Daniel <
Daniel.Christian@honeywell.com> wrote:

>  To be clear, I need sub-processes that are independent of thrift.****
>
> ** **
>
> My solution was to use the threaded server and just monitor each
> sub-processes in a separate thread.  Each thread is then free to block on
> sub-process IO.****
>
> ** **
>
> -Dan****
>
> ** **
>
> ** **
>
> *From:* dvirsky@gmail.com [mailto:dvirsky@gmail.com] *On Behalf Of *Dvir
> Volk
> *Sent:* Monday, February 27, 2012 6:30 AM
> *To:* user@thrift.apache.org; Christian, Daniel
>
> *Subject:* Re: How to have a server manage sub-processes?****
>
>  ** **
>
> Hi Dan, I don't know if it's still relevant after all this time, but
> thrift for python has 2 options to handle your needs (sort of).****
>
> There are 2 server classes you can use - TForkingServer that can fork a
> sub process for each new request, or TProcessPoolServer that spawns a pool
> of "worker" processes that accept requests from the same port (but can't
> directly share state).****
>
> ** **
>
> ** **
>
> On Fri, Jan 27, 2012 at 11:11 PM, Christian, Daniel <
> Daniel.Christian@honeywell.com> wrote:****
>
> My server process needs to spawn some sub processes and interact with them
> occasionally over stdin/stdout (and notice if they die).
>
> What's the  best way to do that with thrift?
>
> My server is in python on Linux.  I'm using thrift 0.8.
>
> Thanks,
> -Dan****
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Dvir Volk****
>
> System Architect, The Everything Project (formerly DoAT)****
>
> http://everything.me****
>
> ** **
>



-- 
Dvir Volk
System Architect, The Everything Project (formerly DoAT)
http://everything.me

RE: How to have a server manage sub-processes?

Posted by "Christian, Daniel" <Da...@Honeywell.com>.
To be clear, I need sub-processes that are independent of thrift.

My solution was to use the threaded server and just monitor each sub-processes in a separate thread.  Each thread is then free to block on sub-process IO.

-Dan


From: dvirsky@gmail.com [mailto:dvirsky@gmail.com] On Behalf Of Dvir Volk
Sent: Monday, February 27, 2012 6:30 AM
To: user@thrift.apache.org; Christian, Daniel
Subject: Re: How to have a server manage sub-processes?

Hi Dan, I don't know if it's still relevant after all this time, but thrift for python has 2 options to handle your needs (sort of).
There are 2 server classes you can use - TForkingServer that can fork a sub process for each new request, or TProcessPoolServer that spawns a pool of "worker" processes that accept requests from the same port (but can't directly share state).


On Fri, Jan 27, 2012 at 11:11 PM, Christian, Daniel <Da...@honeywell.com>> wrote:
My server process needs to spawn some sub processes and interact with them occasionally over stdin/stdout (and notice if they die).

What's the  best way to do that with thrift?

My server is in python on Linux.  I'm using thrift 0.8.

Thanks,
-Dan



--
Dvir Volk
System Architect, The Everything Project (formerly DoAT)
http://everything.me


Re: How to have a server manage sub-processes?

Posted by Dvir Volk <dv...@doat.com>.
Hi Dan, I don't know if it's still relevant after all this time, but thrift
for python has 2 options to handle your needs (sort of).
There are 2 server classes you can use - TForkingServer that can fork a sub
process for each new request, or TProcessPoolServer that spawns a pool of
"worker" processes that accept requests from the same port (but can't
directly share state).


On Fri, Jan 27, 2012 at 11:11 PM, Christian, Daniel <
Daniel.Christian@honeywell.com> wrote:

> My server process needs to spawn some sub processes and interact with them
> occasionally over stdin/stdout (and notice if they die).
>
> What's the  best way to do that with thrift?
>
> My server is in python on Linux.  I'm using thrift 0.8.
>
> Thanks,
> -Dan
>
>


-- 
Dvir Volk
System Architect, The Everything Project (formerly DoAT)
http://everything.me