You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Rush Manbert <ru...@manbert.com> on 2008/12/14 18:04:20 UTC
Multiple server instances question
I posted a while ago asking for opinions about implementing a publish/
subscribe interface using Thrift. I'm going forward with it, but
there's an issue that I don't know how to solve and I'm hoping someone
here can help.
I have a database server with multiple clients. The server and clients
are all on the same machine. The client/server interface uses Thrift
with the default port address of 9090 that the code generator inserted.
Now I want my clients to be able to subscribe to notifications about
changes to the database. I want to use a Thrift interface, which means
that each of my clients will be a Thrift server for the purposes of
receiving notifications. That seems straightforward, but I don't
understand how to have multiple instances of that server running
concurrently on a single machine. I assume this is possible if they
are on different ports, but how do you make that happen with the
Thrift code?
I can imagine a sequence something like this:
* Client instantiates its notification server interface using port
number 0, so the OS assigns a port
* Client somehow retrieves the actual port number (Don't know how to
do that)
* Alternatively: Client selects a port number using some algorithm
I don't know and uses that port
number to instantiate the notification server interface. That
avoids the problem of retrieving
the port number that was assigned by the OS. But it seems a lot
easier to let the OS do the port
selection, since my clients are all independent and would need
to deal with port number collisions.
* Client sends a subscription message to the database server. The
message specifies what the client wants to be told about, and tells
the server the port number.
* Whenever the database server needs to send a notification to this
particular client, he communicates on the specified port. (Don't know
how to do that.)
* Other clients will use other ports, so all notification messages go
to a specific listener.
I'm sorry if this seems like a dumb question. I don't see in my
generated code for the client side of the database server interface
where it knows to talk to port 9090. There appears to be some
networking magic going on under the hood, and I have never written any
networking code, so I don't really know what I'm looking for in the
Thrift sources.
Any help would be greatly appreciated.
- Rush
Re: Multiple server instances question
Posted by Rush Manbert <ru...@manbert.com>.
On Dec 16, 2008, at 1:24 AM, Esteve Fernandez wrote:
> On Monday 15 December 2008 21:25:00 Ben Taitelbaum wrote:
>> While this sounds like it'll work, there's a lot of work (and little
>> details) with maintaining the connections and serving all the
>> clients.
>> Why not just use a message queue (or multiple topics) that the
>> clients
>> can subscribe to. Then the server can publish simple messages
>> indicating that an event took place. These could either be very basic
>> messages with the intent of the clients querying the server for more
>> info, or they could be serialized thrift structs containing info
>> about
>> the entire event.
>>
>> If you go the way you're planning though, why not make it support a
>> STOMP protocol and release it as a thrift message queue?
>
> Or (if you're using Python) use txAMQP [1] right now :-) We added
> support for
> Thrift RPC on top of AMQP in the 0.2 version. The only drawback is
> that you
> have to patch the Thrift compiler [2], but if you're using a Debian-
> based
> Linux distro, you can use our Debian packages [3]
>
> We plan to do the same for Java/MINA (and maybe for Ruby/
> EventMachine and
> C++/ASIO), but can't give a date.
>
> On a side note, I just received the confirmation that I'll give a
> talk about
> Thrift, Twisted and AMQP at US PyCon 2009.
>
> Cheers.
>
> 1 - https://launchpad.net/txamqp
> 2 - https://issues.apache.org/jira/browse/THRIFT-148
> 3 - https://launchpad.net/~txamqpteam/+archive
Thanks for the reply. Not using Python at present.
Best regards,
Rush
Re: Multiple server instances question
Posted by Esteve Fernandez <es...@sindominio.net>.
On Monday 15 December 2008 21:25:00 Ben Taitelbaum wrote:
> While this sounds like it'll work, there's a lot of work (and little
> details) with maintaining the connections and serving all the clients.
> Why not just use a message queue (or multiple topics) that the clients
> can subscribe to. Then the server can publish simple messages
> indicating that an event took place. These could either be very basic
> messages with the intent of the clients querying the server for more
> info, or they could be serialized thrift structs containing info about
> the entire event.
>
> If you go the way you're planning though, why not make it support a
> STOMP protocol and release it as a thrift message queue?
Or (if you're using Python) use txAMQP [1] right now :-) We added support for
Thrift RPC on top of AMQP in the 0.2 version. The only drawback is that you
have to patch the Thrift compiler [2], but if you're using a Debian-based
Linux distro, you can use our Debian packages [3]
We plan to do the same for Java/MINA (and maybe for Ruby/EventMachine and
C++/ASIO), but can't give a date.
On a side note, I just received the confirmation that I'll give a talk about
Thrift, Twisted and AMQP at US PyCon 2009.
Cheers.
1 - https://launchpad.net/txamqp
2 - https://issues.apache.org/jira/browse/THRIFT-148
3 - https://launchpad.net/~txamqpteam/+archive
Re: Multiple server instances question
Posted by Rush Manbert <ru...@manbert.com>.
I have owed you a reply for quite a while now. But I have been really
busy implementing the database commits from my Thrift-protocol caching
clients. (The same clients who will want to sign up for database
change notifications.)
I have at least one use case where the connection is between machines.
For that reason, I want to implement something that works identically
when the client and server are on the same machine or on different
machines. That rules out message queues for me. The fact that the
Thrift library implements all the messaging is a big plus. I also
eventually need to do this on Windows, but that's another story.
I looked at STOMP. This might sound strange, since we are obviously
putting a database on our end user systems, and a database server
daemon, but the requirement to have a STOMP message broker on the end
user machine is a deal breaker for us. We don't have much control over
the end user machine, and we need to install as little software as
possible there.
I did solve my multiple servers problem, but will need something
better for the publish/subscribe interface. I probably have a small
enough number of potential clients that I can use a fixed set of port
numbers for use by the subscribers. I'd rather have a more general-
case solution, but I don't see one at present.
Best regards,
Rush
On Dec 15, 2008, at 12:25 PM, Ben Taitelbaum wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> While this sounds like it'll work, there's a lot of work (and little
> details) with maintaining the connections and serving all the
> clients. Why not just use a message queue (or multiple topics) that
> the clients can subscribe to. Then the server can publish simple
> messages indicating that an event took place. These could either be
> very basic messages with the intent of the clients querying the
> server for more info, or they could be serialized thrift structs
> containing info about the entire event.
>
> If you go the way you're planning though, why not make it support a
> STOMP protocol and release it as a thrift message queue?
>
> - -Ben
>
> On Dec 14, 2008, at 12:04 PM, Rush Manbert wrote:
>
>> I posted a while ago asking for opinions about implementing a
>> publish/subscribe interface using Thrift. I'm going forward with
>> it, but there's an issue that I don't know how to solve and I'm
>> hoping someone here can help.
>>
>> I have a database server with multiple clients. The server and
>> clients are all on the same machine. The client/server interface
>> uses Thrift with the default port address of 9090 that the code
>> generator inserted.
>>
>> Now I want my clients to be able to subscribe to notifications
>> about changes to the database. I want to use a Thrift interface,
>> which means that each of my clients will be a Thrift server for the
>> purposes of receiving notifications. That seems straightforward,
>> but I don't understand how to have multiple instances of that
>> server running concurrently on a single machine. I assume this is
>> possible if they are on different ports, but how do you make that
>> happen with the Thrift code?
>>
>> I can imagine a sequence something like this:
>> * Client instantiates its notification server interface using port
>> number 0, so the OS assigns a port
>> * Client somehow retrieves the actual port number (Don't know how
>> to do that)
>> * Alternatively: Client selects a port number using some
>> algorithm I don't know and uses that port
>> number to instantiate the notification server interface. That
>> avoids the problem of retrieving
>> the port number that was assigned by the OS. But it seems a lot
>> easier to let the OS do the port
>> selection, since my clients are all independent and would need
>> to deal with port number collisions.
>> * Client sends a subscription message to the database server. The
>> message specifies what the client wants to be told about, and tells
>> the server the port number.
>> * Whenever the database server needs to send a notification to this
>> particular client, he communicates on the specified port. (Don't
>> know how to do that.)
>> * Other clients will use other ports, so all notification messages
>> go to a specific listener.
>>
>> I'm sorry if this seems like a dumb question. I don't see in my
>> generated code for the client side of the database server interface
>> where it knows to talk to port 9090. There appears to be some
>> networking magic going on under the hood, and I have never written
>> any networking code, so I don't really know what I'm looking for in
>> the Thrift sources.
>>
>> Any help would be greatly appreciated.
>>
>> - Rush
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iEYEARECAAYFAklGvRwACgkQcbwI7gfD0FUXmACgsgsOyFEoXt6yfWZiv5JkgLtE
> VA0An1TpUJVHK6kBGz0hJFgguqL19asC
> =XRf7
> -----END PGP SIGNATURE-----
Re: Multiple server instances question
Posted by Ben Taitelbaum <bt...@cs.oberlin.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
While this sounds like it'll work, there's a lot of work (and little
details) with maintaining the connections and serving all the clients.
Why not just use a message queue (or multiple topics) that the clients
can subscribe to. Then the server can publish simple messages
indicating that an event took place. These could either be very basic
messages with the intent of the clients querying the server for more
info, or they could be serialized thrift structs containing info about
the entire event.
If you go the way you're planning though, why not make it support a
STOMP protocol and release it as a thrift message queue?
- -Ben
On Dec 14, 2008, at 12:04 PM, Rush Manbert wrote:
> I posted a while ago asking for opinions about implementing a
> publish/subscribe interface using Thrift. I'm going forward with it,
> but there's an issue that I don't know how to solve and I'm hoping
> someone here can help.
>
> I have a database server with multiple clients. The server and
> clients are all on the same machine. The client/server interface
> uses Thrift with the default port address of 9090 that the code
> generator inserted.
>
> Now I want my clients to be able to subscribe to notifications about
> changes to the database. I want to use a Thrift interface, which
> means that each of my clients will be a Thrift server for the
> purposes of receiving notifications. That seems straightforward, but
> I don't understand how to have multiple instances of that server
> running concurrently on a single machine. I assume this is possible
> if they are on different ports, but how do you make that happen with
> the Thrift code?
>
> I can imagine a sequence something like this:
> * Client instantiates its notification server interface using port
> number 0, so the OS assigns a port
> * Client somehow retrieves the actual port number (Don't know how to
> do that)
> * Alternatively: Client selects a port number using some
> algorithm I don't know and uses that port
> number to instantiate the notification server interface. That
> avoids the problem of retrieving
> the port number that was assigned by the OS. But it seems a lot
> easier to let the OS do the port
> selection, since my clients are all independent and would need
> to deal with port number collisions.
> * Client sends a subscription message to the database server. The
> message specifies what the client wants to be told about, and tells
> the server the port number.
> * Whenever the database server needs to send a notification to this
> particular client, he communicates on the specified port. (Don't
> know how to do that.)
> * Other clients will use other ports, so all notification messages
> go to a specific listener.
>
> I'm sorry if this seems like a dumb question. I don't see in my
> generated code for the client side of the database server interface
> where it knows to talk to port 9090. There appears to be some
> networking magic going on under the hood, and I have never written
> any networking code, so I don't really know what I'm looking for in
> the Thrift sources.
>
> Any help would be greatly appreciated.
>
> - Rush
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
iEYEARECAAYFAklGvRwACgkQcbwI7gfD0FUXmACgsgsOyFEoXt6yfWZiv5JkgLtE
VA0An1TpUJVHK6kBGz0hJFgguqL19asC
=XRf7
-----END PGP SIGNATURE-----
Re: Multiple server instances question
Posted by Rush Manbert <ru...@manbert.com>.
As far as I can see, I will need a different port number for each
subscriber that is on the local machine. The general case problem is a
PITA to solve, it seems, but I probably don't realistically have very
many potential clients. We can probably pre-assign a set of port
numbers and let the database server dole them out to the clients. I'll
still need to handle subscribers who die while holding a port, but
that doesn't seem to be too hard. I already had to solve that problem
for stale database commit locks.
The point about firewalls and monitoring tools was much appreciated.
Randomness doesn't help when you're trying to debug. :-)
Best regards,
Rush
On Dec 15, 2008, at 6:12 PM, Mark Slee wrote:
> Sounds correct. If you're building a specific service though, I'd
> recomment just choosing a port # for your service ahead of time, and
> using that. Keeps things much more manageable on the networking
> side, allows you to add on firewall rules etc. later if the need
> arises, and you shouldn't run into any conflicts as long as you're
> using an unreserved high port number.
>
> Using randomly assigned ports will make your traffic a little harder
> to monitor or identify using off-the-shelf tools.
>
> -----Original Message-----
> From: Rush Manbert [mailto:rush@manbert.com]
> Sent: Monday, December 15, 2008 10:20 AM
> To: thrift-dev@incubator.apache.org
> Subject: Re: Multiple server instances question
>
> Okay, once again I make a fool of myself on a public mailing list. I
> swear I'm really not that stupid...
>
> I took another look at my database client side code and now I know
> where I set port number 9090 on the client side of the interface.
> Obviously I have complete control and I can match the notification
> side to whatever the database client tells it.
>
> It looks like I can specify port 0 when I instantiate the TSocket,
> then call getPort() to find out the port that the OS gave me. Does
> that sound right?
>
> Sheepishly,
> Rush
>
> On Dec 14, 2008, at 9:04 AM, Rush Manbert wrote:
>
>> I posted a while ago asking for opinions about implementing a
>> publish/subscribe interface using Thrift. I'm going forward with it,
>> but there's an issue that I don't know how to solve and I'm hoping
>> someone here can help.
>>
>> I have a database server with multiple clients. The server and
>> clients
>> are all on the same machine. The client/server interface uses Thrift
>> with the default port address of 9090 that the code generator
>> inserted.
>>
>> Now I want my clients to be able to subscribe to notifications about
>> changes to the database. I want to use a Thrift interface, which
>> means
>> that each of my clients will be a Thrift server for the purposes of
>> receiving notifications. That seems straightforward, but I don't
>> understand how to have multiple instances of that server running
>> concurrently on a single machine. I assume this is possible if they
>> are on different ports, but how do you make that happen with the
>> Thrift code?
>>
>> I can imagine a sequence something like this:
>> * Client instantiates its notification server interface using port
>> number 0, so the OS assigns a port
>> * Client somehow retrieves the actual port number (Don't know how to
>> do that)
>> * Alternatively: Client selects a port number using some
>> algorithm I don't know and uses that port
>> number to instantiate the notification server interface. That
>> avoids the problem of retrieving
>> the port number that was assigned by the OS. But it seems a lot
>> easier to let the OS do the port
>> selection, since my clients are all independent and would need to
>> deal with port number collisions.
>> * Client sends a subscription message to the database server. The
>> message specifies what the client wants to be told about, and tells
>> the server the port number.
>> * Whenever the database server needs to send a notification to this
>> particular client, he communicates on the specified port. (Don't know
>> how to do that.)
>> * Other clients will use other ports, so all notification messages go
>> to a specific listener.
>>
>> I'm sorry if this seems like a dumb question. I don't see in my
>> generated code for the client side of the database server interface
>> where it knows to talk to port 9090. There appears to be some
>> networking magic going on under the hood, and I have never written
>> any
>> networking code, so I don't really know what I'm looking for in the
>> Thrift sources.
>>
>> Any help would be greatly appreciated.
>>
>> - Rush
RE: Multiple server instances question
Posted by Mark Slee <ms...@facebook.com>.
Sounds correct. If you're building a specific service though, I'd recomment just choosing a port # for your service ahead of time, and using that. Keeps things much more manageable on the networking side, allows you to add on firewall rules etc. later if the need arises, and you shouldn't run into any conflicts as long as you're using an unreserved high port number.
Using randomly assigned ports will make your traffic a little harder to monitor or identify using off-the-shelf tools.
-----Original Message-----
From: Rush Manbert [mailto:rush@manbert.com]
Sent: Monday, December 15, 2008 10:20 AM
To: thrift-dev@incubator.apache.org
Subject: Re: Multiple server instances question
Okay, once again I make a fool of myself on a public mailing list. I swear I'm really not that stupid...
I took another look at my database client side code and now I know where I set port number 9090 on the client side of the interface.
Obviously I have complete control and I can match the notification side to whatever the database client tells it.
It looks like I can specify port 0 when I instantiate the TSocket, then call getPort() to find out the port that the OS gave me. Does that sound right?
Sheepishly,
Rush
On Dec 14, 2008, at 9:04 AM, Rush Manbert wrote:
> I posted a while ago asking for opinions about implementing a
> publish/subscribe interface using Thrift. I'm going forward with it,
> but there's an issue that I don't know how to solve and I'm hoping
> someone here can help.
>
> I have a database server with multiple clients. The server and clients
> are all on the same machine. The client/server interface uses Thrift
> with the default port address of 9090 that the code generator
> inserted.
>
> Now I want my clients to be able to subscribe to notifications about
> changes to the database. I want to use a Thrift interface, which means
> that each of my clients will be a Thrift server for the purposes of
> receiving notifications. That seems straightforward, but I don't
> understand how to have multiple instances of that server running
> concurrently on a single machine. I assume this is possible if they
> are on different ports, but how do you make that happen with the
> Thrift code?
>
> I can imagine a sequence something like this:
> * Client instantiates its notification server interface using port
> number 0, so the OS assigns a port
> * Client somehow retrieves the actual port number (Don't know how to
> do that)
> * Alternatively: Client selects a port number using some
> algorithm I don't know and uses that port
> number to instantiate the notification server interface. That
> avoids the problem of retrieving
> the port number that was assigned by the OS. But it seems a lot
> easier to let the OS do the port
> selection, since my clients are all independent and would need to
> deal with port number collisions.
> * Client sends a subscription message to the database server. The
> message specifies what the client wants to be told about, and tells
> the server the port number.
> * Whenever the database server needs to send a notification to this
> particular client, he communicates on the specified port. (Don't know
> how to do that.)
> * Other clients will use other ports, so all notification messages go
> to a specific listener.
>
> I'm sorry if this seems like a dumb question. I don't see in my
> generated code for the client side of the database server interface
> where it knows to talk to port 9090. There appears to be some
> networking magic going on under the hood, and I have never written any
> networking code, so I don't really know what I'm looking for in the
> Thrift sources.
>
> Any help would be greatly appreciated.
>
> - Rush
Re: Multiple server instances question
Posted by Rush Manbert <ru...@manbert.com>.
Okay, once again I make a fool of myself on a public mailing list. I
swear I'm really not that stupid...
I took another look at my database client side code and now I know
where I set port number 9090 on the client side of the interface.
Obviously I have complete control and I can match the notification
side to whatever the database client tells it.
It looks like I can specify port 0 when I instantiate the TSocket,
then call getPort() to find out the port that the OS gave me. Does
that sound right?
Sheepishly,
Rush
On Dec 14, 2008, at 9:04 AM, Rush Manbert wrote:
> I posted a while ago asking for opinions about implementing a
> publish/subscribe interface using Thrift. I'm going forward with it,
> but there's an issue that I don't know how to solve and I'm hoping
> someone here can help.
>
> I have a database server with multiple clients. The server and
> clients are all on the same machine. The client/server interface
> uses Thrift with the default port address of 9090 that the code
> generator inserted.
>
> Now I want my clients to be able to subscribe to notifications about
> changes to the database. I want to use a Thrift interface, which
> means that each of my clients will be a Thrift server for the
> purposes of receiving notifications. That seems straightforward, but
> I don't understand how to have multiple instances of that server
> running concurrently on a single machine. I assume this is possible
> if they are on different ports, but how do you make that happen with
> the Thrift code?
>
> I can imagine a sequence something like this:
> * Client instantiates its notification server interface using port
> number 0, so the OS assigns a port
> * Client somehow retrieves the actual port number (Don't know how to
> do that)
> * Alternatively: Client selects a port number using some
> algorithm I don't know and uses that port
> number to instantiate the notification server interface. That
> avoids the problem of retrieving
> the port number that was assigned by the OS. But it seems a lot
> easier to let the OS do the port
> selection, since my clients are all independent and would need
> to deal with port number collisions.
> * Client sends a subscription message to the database server. The
> message specifies what the client wants to be told about, and tells
> the server the port number.
> * Whenever the database server needs to send a notification to this
> particular client, he communicates on the specified port. (Don't
> know how to do that.)
> * Other clients will use other ports, so all notification messages
> go to a specific listener.
>
> I'm sorry if this seems like a dumb question. I don't see in my
> generated code for the client side of the database server interface
> where it knows to talk to port 9090. There appears to be some
> networking magic going on under the hood, and I have never written
> any networking code, so I don't really know what I'm looking for in
> the Thrift sources.
>
> Any help would be greatly appreciated.
>
> - Rush