Notification Strategy

[Rush Manbert <ru...@manbert.com>]

We have a Thrift server application that manages data on behalf of  
numerous clients. We would like to add the capability for clients to  
be notified when the managed data is changed. Clients would tell the  
server what data they care about, and would be notified asynchronously  
if the data changes.

Clients talk to the server through an object interface that we have  
layered on top of the Thrift client side interface. We were thinking  
that we could add the notifications capability by adding a Thrift  
server to the client side object. This server would listen for  
notification messages on a runtime-selected port address. He would  
register the port address with the managed data server when the client  
side object is instantiated.

We like this approach because it means all of our client/server  
interactions use a Thrift interface. It lets us easily have clients be  
remote to the server.

My question is whether this seems like a good approach to the list  
readers and the Thrift developers. Is there some mechanism already in  
place within the Thrift library for this? Is there some other better  
approach?

Thanks,
Rush

Re: Notification Strategy

[Luke Lu <th...@vicaya.com>]

On Nov 10, 2008, at 2:28 PM, Rush Manbert wrote:

> We have a Thrift server application that manages data on behalf of  
> numerous clients. We would like to add the capability for clients to  
> be notified when the managed data is changed. Clients would tell the  
> server what data they care about, and would be notified  
> asynchronously if the data changes.

You're talking about a classic pub/sub architecture here.

> Clients talk to the server through an object interface that we have  
> layered on top of the Thrift client side interface. We were thinking  
> that we could add the notifications capability by adding a Thrift  
> server to the client side object. This server would listen for  
> notification messages on a runtime-selected port address. He would  
> register the port address with the managed data server when the  
> client side object is instantiated.
>
> We like this approach because it means all of our client/server  
> interactions use a Thrift interface. It lets us easily have clients  
> be remote to the server.

You don't need to listen on random ports at client side, which can  
cause all sorts of problems if there is any firewall in between. You  
can have a subscription/topic manager at the server side to manage  
connections from the clients.

> My question is whether this seems like a good approach to the list  
> readers and the Thrift developers. Is there some mechanism already  
> in place within the Thrift library for this? Is there some other  
> better approach?

I don't think there is an equivalent of subscription/topic manager in  
Thrift. The problem has been solved comprehensively before and  
elsewhere though. See IceStorm for some ideas:

http://zeroc.com/doc/Ice-3.3.0-IceTouch/manual/IceStorm.html

__Luke

Re: Notification Strategy

[Carl Byström <ca...@esportnetwork.com>]

On Mon, Nov 10, 2008 at 11:28 PM, Rush Manbert <ru...@manbert.com> wrote:

> My question is whether this seems like a good approach to the list readers
> and the Thrift developers. Is there some mechanism already in place within
> the Thrift library for this? Is there some other better approach?
>
> Thanks,
> Rush
>

We have a system in place, based on STOMP (http://stomp.codehaus.org/). It's
a message-oriented protocol in which we wrap Thrift-serialized structures.
On top of this you have the possibility to implement RPC or leave it as a
message-passing system with Thrift structures.

/Carl

Re: Notification Strategy

[Rush Manbert <ru...@manbert.com>]

Thanks to Mark, Carl, Luke, and Esteve for your replies. I'm going to  
go check out the links you provided and make a recommendation to my  
colleagues here.

Best regards,
Rush

Re: Notification Strategy

[Esteve Fernandez <es...@sindominio.net>]

On Tuesday 11 November 2008 03:56:32 Mark Slee wrote:
> This sounds reasonable to me and is probably how I would design such a
> system. Thrift clients/servers are fundamentally unidirectional RPC, not a
> bidirectional messaging system, so having a server on each end makes the
> most sense. It sounds like what you really want here is asynchronous
> messaging library, which isn't Thrift's primary intended purpose but can
> certainly be accomplished.

Well, if you're using Python, you may want to take a look at txAMQP [1] It 
already implements asynchronous messaging using AMQP as the underlying 
transport. The idea is fairly simple, it simply wraps Thrift calls in an AMQP 
message. We plan to eventually release a Java port, but it should be pretty 
straightforward to port it to other languages.

Cheers.

1 - http://launchpad.net/txamqp

RE: Notification Strategy

[Mark Slee <ms...@facebook.com>]

This sounds reasonable to me and is probably how I would design such a system. Thrift clients/servers are fundamentally unidirectional RPC, not a bidirectional messaging system, so having a server on each end makes the most sense. It sounds like what you really want here is asynchronous messaging library, which isn't Thrift's primary intended purpose but can certainly be accomplished.

One other technical possibility is to borrow from the COMET approach:
http://en.wikipedia.org/wiki/Comet_(programming)

Basically, you could use long timeouts and leave a client request open. This, however, is pretty hackneyed and I would only suggest this approach if there were some reason why you did not want to have server ports open on the client. But it sounds like you're fine with that.

Hope that helps... Let us know how it goes.

Cheers,
Mark

-----Original Message-----
From: Rush Manbert [mailto:rush@manbert.com]
Sent: Monday, November 10, 2008 2:28 PM
To: thrift-dev@incubator.apache.org
Subject: Notification Strategy

We have a Thrift server application that manages data on behalf of numerous clients. We would like to add the capability for clients to be notified when the managed data is changed. Clients would tell the server what data they care about, and would be notified asynchronously if the data changes.

Clients talk to the server through an object interface that we have layered on top of the Thrift client side interface. We were thinking that we could add the notifications capability by adding a Thrift server to the client side object. This server would listen for notification messages on a runtime-selected port address. He would register the port address with the managed data server when the client side object is instantiated.

We like this approach because it means all of our client/server interactions use a Thrift interface. It lets us easily have clients be remote to the server.

My question is whether this seems like a good approach to the list readers and the Thrift developers. Is there some mechanism already in place within the Thrift library for this? Is there some other better approach?

Thanks,
Rush