Thrift async capabailities

[MSR <ms...@gmail.com>]

Hi,

I am trying to get information about thrift capabilities and see if it is a
good fit for our project. I have been trying to read documentation and
mailing lists.

I have a few question.s and really appreciate if someone can answer them.

 Lets say I have two services,

Foo1();
Foo2();

On The server Side:(C++)
1. I want to be able to run on Single Port (for both services) and support
large (thousands) number clients. Is this possible?
2. I want thread support. i.e. I want both Foo1 and Foo2 to run
simultaneously on different threads.
3. Can I send responses asynchronously at much later time? i.e. lets say
Foo1 needs to talk to some other server to get the answer. Obviously, I
don't want my thread to be blocked on that. I would like to be able send the
response back at a later without blocking anywhere.
4. Can there be multiple pending calls for the same service from the same
client. My code can differentiate which call came from which client.
5.On the same note, can responses be sent out of order?


On the client side(C#)
1. Can I use one connection to request multiple services?
2. Similar to question 4 above. Can the client make multiple pending RPC
calls on the same connection (My code will know which response corresponds
to which request).


Thanks,
MSR

Re: Thrift async capabailities

[Joel Meyer <jo...@gmail.com>]

On Sat, Feb 13, 2010 at 11:40 AM, MSR <ms...@gmail.com> wrote:

> Thanks Bryan. comments inlne.
>
> On Sat, Feb 13, 2010 at 1:24 PM, Bryan Duxbury <br...@rapleaf.com> wrote:
>
> > On Fri, Feb 12, 2010 at 10:03 PM, MSR <ms...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I am trying to get information about thrift capabilities and see if it
> is
> > a
> > > good fit for our project. I have been trying to read documentation and
> > > mailing lists.
> > >
> > > I have a few question.s and really appreciate if someone can answer
> them.
> > >
> > >  Lets say I have two services,
> > >
> > > Foo1();
> > > Foo2();
> > >
> > > On The server Side:(C++)
> > > 1. I want to be able to run on Single Port (for both services) and
> > support
> > > large (thousands) number clients. Is this possible?
> > >
> > I believe that someone has done some work on multiplexing services on a
> > single port, but I wouldn't bet that functionality is available. Is there
> > any reason that you *must* have two separate service definitions and
> > exactly
> > one port? Your life will be easier if you can combine the services or use
> > two ports.
> >
> > Thousands of clients are no problem, if your server hardware can handle
> > that
> > kind of load.
> >
>
> Yeah. My server can handle the load. As of today, I don't really need to
> multiplex services on the same port. I just wanted to know if I really need
> to open all those ports or not.
>
> >
> >
> > > 2. I want thread support. i.e. I want both Foo1 and Foo2 to run
> > > simultaneously on different threads.
> > >
> > The internal thread model of your server's application logic is what
> > determines this, combined with the Thrift server implementation you use.
> I
> > don't have firsthand c++ experience, but in Java we have servers that are
> > explicitly single-threaded, multi-threaded, and thread pooled. I am
> certain
> > c++ has some of the same set of servers.
> >
> Great. There is not much documentation through to figure out which  server
> implementation to use. I read the ThreadPool Server only support 4
> connections. I really want to have 1000s of connections. I am guessing
> NonBlockingServer but not clear to me how I use it with multiple threads.
>
> >
> >
> > > 3. Can I send responses asynchronously at much later time? i.e. lets
> say
> > > Foo1 needs to talk to some other server to get the answer. Obviously, I
> > > don't want my thread to be blocked on that. I would like to be able
> send
> > > the
> > > response back at a later without blocking anywhere.
> > >
> > Thrift does not directly support "asynchronous" responses. Your server
> can
> > block as long as the client is willing to wait to return a response,
> > though.
> > If you don't want the client to have to wait, then you should implement a
> > separate method on your service that lets the client check back in for
> > updates.
> >
> This is very discouraging to me. My scenario is such that we have hierarchy
> of services. So no body in that hierarchy can block or afford to have a
> thread per connection.  Corrent me if I am wrong, but I think a polling
> kind
> of mechanism to find out if the request is completed  can be inefficient,
> increase latency and potentially really messy (when should the client check
> for results? how long should the server hold the results for a request
> etc....)
>

If you only use async/one-way calls and have threads on both ends doing
reading, then yes, you can achieve something like this:
http://joelpm.com/2009/04/03/thrift-bidirectional-async-rpc.html

I wouldn't go so far as to say it's a good idea to implement that type of
behavior using Thrift, but it is at least (somewhat) possible if you can
live with certain constraints.


>
> >
> >
> > > 4. Can there be multiple pending calls for the same service from the
> same
> > > client. My code can differentiate which call came from which client.
> > >
> > If you want to be able to send more than one request from the client
> before
> > getting a response, you should use "oneway" methods a separate method or
> > methods to check back in for responses later. Thrift does not currently
> > implement a general message-passing type of communication like you
> > describe.
> >
> >
> > > 5.On the same note, can responses be sent out of order?
> > >
> > Again, the client is what dictates the order responses will be received,
> if
> > you are using the "check back" approach.
> >
> >
> >
> > >
> > >
> > > On the client side(C#)
> > > 1. Can I use one connection to request multiple services?
> > >
> > This depends on whether that aforementioned multiplexing work was ever
> > completed or not. I would tend to believe no.
> >
> >
> > > 2. Similar to question 4 above. Can the client make multiple pending
> RPC
> > > calls on the same connection (My code will know which response
> > corresponds
> > > to which request).
> > >
> > Yes, if you use oneway methods and a check back approach.
> >
> > -Bryan
> >
>

Re: Thrift async capabailities

[MSR <ms...@gmail.com>]

Thanks Bryan. comments inlne.

On Sat, Feb 13, 2010 at 1:24 PM, Bryan Duxbury <br...@rapleaf.com> wrote:

> On Fri, Feb 12, 2010 at 10:03 PM, MSR <ms...@gmail.com> wrote:
>
> > Hi,
> >
> > I am trying to get information about thrift capabilities and see if it is
> a
> > good fit for our project. I have been trying to read documentation and
> > mailing lists.
> >
> > I have a few question.s and really appreciate if someone can answer them.
> >
> >  Lets say I have two services,
> >
> > Foo1();
> > Foo2();
> >
> > On The server Side:(C++)
> > 1. I want to be able to run on Single Port (for both services) and
> support
> > large (thousands) number clients. Is this possible?
> >
> I believe that someone has done some work on multiplexing services on a
> single port, but I wouldn't bet that functionality is available. Is there
> any reason that you *must* have two separate service definitions and
> exactly
> one port? Your life will be easier if you can combine the services or use
> two ports.
>
> Thousands of clients are no problem, if your server hardware can handle
> that
> kind of load.
>

Yeah. My server can handle the load. As of today, I don't really need to
multiplex services on the same port. I just wanted to know if I really need
to open all those ports or not.

>
>
> > 2. I want thread support. i.e. I want both Foo1 and Foo2 to run
> > simultaneously on different threads.
> >
> The internal thread model of your server's application logic is what
> determines this, combined with the Thrift server implementation you use. I
> don't have firsthand c++ experience, but in Java we have servers that are
> explicitly single-threaded, multi-threaded, and thread pooled. I am certain
> c++ has some of the same set of servers.
>
Great. There is not much documentation through to figure out which  server
implementation to use. I read the ThreadPool Server only support 4
connections. I really want to have 1000s of connections. I am guessing
NonBlockingServer but not clear to me how I use it with multiple threads.

>
>
> > 3. Can I send responses asynchronously at much later time? i.e. lets say
> > Foo1 needs to talk to some other server to get the answer. Obviously, I
> > don't want my thread to be blocked on that. I would like to be able send
> > the
> > response back at a later without blocking anywhere.
> >
> Thrift does not directly support "asynchronous" responses. Your server can
> block as long as the client is willing to wait to return a response,
> though.
> If you don't want the client to have to wait, then you should implement a
> separate method on your service that lets the client check back in for
> updates.
>
This is very discouraging to me. My scenario is such that we have hierarchy
of services. So no body in that hierarchy can block or afford to have a
thread per connection.  Corrent me if I am wrong, but I think a polling kind
of mechanism to find out if the request is completed  can be inefficient,
increase latency and potentially really messy (when should the client check
for results? how long should the server hold the results for a request
etc....)

>
>
> > 4. Can there be multiple pending calls for the same service from the same
> > client. My code can differentiate which call came from which client.
> >
> If you want to be able to send more than one request from the client before
> getting a response, you should use "oneway" methods a separate method or
> methods to check back in for responses later. Thrift does not currently
> implement a general message-passing type of communication like you
> describe.
>
>
> > 5.On the same note, can responses be sent out of order?
> >
> Again, the client is what dictates the order responses will be received, if
> you are using the "check back" approach.
>
>
>
> >
> >
> > On the client side(C#)
> > 1. Can I use one connection to request multiple services?
> >
> This depends on whether that aforementioned multiplexing work was ever
> completed or not. I would tend to believe no.
>
>
> > 2. Similar to question 4 above. Can the client make multiple pending RPC
> > calls on the same connection (My code will know which response
> corresponds
> > to which request).
> >
> Yes, if you use oneway methods and a check back approach.
>
> -Bryan
>

Re: Thrift async capabailities

[Bryan Duxbury <br...@rapleaf.com>]

On Fri, Feb 12, 2010 at 10:03 PM, MSR <ms...@gmail.com> wrote:

> Hi,
>
> I am trying to get information about thrift capabilities and see if it is a
> good fit for our project. I have been trying to read documentation and
> mailing lists.
>
> I have a few question.s and really appreciate if someone can answer them.
>
>  Lets say I have two services,
>
> Foo1();
> Foo2();
>
> On The server Side:(C++)
> 1. I want to be able to run on Single Port (for both services) and support
> large (thousands) number clients. Is this possible?
>
I believe that someone has done some work on multiplexing services on a
single port, but I wouldn't bet that functionality is available. Is there
any reason that you *must* have two separate service definitions and exactly
one port? Your life will be easier if you can combine the services or use
two ports.

Thousands of clients are no problem, if your server hardware can handle that
kind of load.


> 2. I want thread support. i.e. I want both Foo1 and Foo2 to run
> simultaneously on different threads.
>
The internal thread model of your server's application logic is what
determines this, combined with the Thrift server implementation you use. I
don't have firsthand c++ experience, but in Java we have servers that are
explicitly single-threaded, multi-threaded, and thread pooled. I am certain
c++ has some of the same set of servers.


> 3. Can I send responses asynchronously at much later time? i.e. lets say
> Foo1 needs to talk to some other server to get the answer. Obviously, I
> don't want my thread to be blocked on that. I would like to be able send
> the
> response back at a later without blocking anywhere.
>
Thrift does not directly support "asynchronous" responses. Your server can
block as long as the client is willing to wait to return a response, though.
If you don't want the client to have to wait, then you should implement a
separate method on your service that lets the client check back in for
updates.


> 4. Can there be multiple pending calls for the same service from the same
> client. My code can differentiate which call came from which client.
>
If you want to be able to send more than one request from the client before
getting a response, you should use "oneway" methods a separate method or
methods to check back in for responses later. Thrift does not currently
implement a general message-passing type of communication like you describe.


> 5.On the same note, can responses be sent out of order?
>
Again, the client is what dictates the order responses will be received, if
you are using the "check back" approach.



>
>
> On the client side(C#)
> 1. Can I use one connection to request multiple services?
>
This depends on whether that aforementioned multiplexing work was ever
completed or not. I would tend to believe no.


> 2. Similar to question 4 above. Can the client make multiple pending RPC
> calls on the same connection (My code will know which response corresponds
> to which request).
>
Yes, if you use oneway methods and a check back approach.

-Bryan