You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by David Bennett <da...@yorkage.com> on 2015/09/14 03:57:31 UTC
Thrift as an in-process server?
Is it possible to use Thrift for in-process cross-language marshalling?
For a mobile or desktop app there is no particular benefit in running a socket-based server, and it's much easier to install and start up an app that makes an in-process call to a dynamic library.
So is this possible?
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org
RE: Thrift as an in-process server?
Posted by David Bennett <da...@yorkage.com>.
Thrift generates different code for each of its target languages, and if a protocol/transport is supported at all it should work the same on every language.
I expect the serialisation overhead: every request you make to Google or Facebook or even ODBC/JDBC already goes through heaps of that.
It just isn't that easy to work out exactly what is built into Thrift other than the default socket interface, and what works across which languages, which is why the question here.
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org
-----Original Message-----
From: Stig Bakken [mailto:stig@zedge.net]
Sent: Tuesday, 15 September 2015 7:41 PM
To: user@thrift.apache.org
Subject: Re: Thrift as an in-process server?
It would be possible, but you must handle compatibility in a way that works with Thrift.
For example, you could have a mechanism for getting an implementation of a service interface from the library, but if that library was built with generated code that does not align with what the (C/C++) program was built with, you may end up with pointers going where you don't want them (feel free to correct me here, anyone more familiar with Thrift's C++ generator).
For more dynamic languages, this technique should work though.
The way Thrift handles compatibility is through the protocol and transport layers, which means you must serialize and deserialize data for every single call. You would need some kind of initialization that creates a transport shared between the shared lib and the main process, and agree on the protocol and service. This transport could use in-process memory to temporarily store serialized parameters and return values (you may be able to just use the memory buffer, but I suspect you'll have to roll your own).
So, possible, yes.
- Stig
On Mon, Sep 14, 2015 at 3:57 AM, David Bennett <da...@yorkage.com> wrote:
> Is it possible to use Thrift for in-process cross-language marshalling?
>
> For a mobile or desktop app there is no particular benefit in running
> a socket-based server, and it's much easier to install and start up an
> app that makes an in-process call to a dynamic library.
>
> So is this possible?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
>
--
Stig Bakken
CTO, Zedge.net - free your phone!
Re: Thrift as an in-process server?
Posted by Stig Bakken <st...@zedge.net>.
It would be possible, but you must handle compatibility in a way that works
with Thrift.
For example, you could have a mechanism for getting an implementation of a
service interface from the library, but if that library was built with
generated code that does not align with what the (C/C++) program was built
with, you may end up with pointers going where you don't want them (feel
free to correct me here, anyone more familiar with Thrift's C++ generator).
For more dynamic languages, this technique should work though.
The way Thrift handles compatibility is through the protocol and transport
layers, which means you must serialize and deserialize data for every
single call. You would need some kind of initialization that creates a
transport shared between the shared lib and the main process, and agree on
the protocol and service. This transport could use in-process memory to
temporarily store serialized parameters and return values (you may be able
to just use the memory buffer, but I suspect you'll have to roll your own).
So, possible, yes.
- Stig
On Mon, Sep 14, 2015 at 3:57 AM, David Bennett <da...@yorkage.com> wrote:
> Is it possible to use Thrift for in-process cross-language marshalling?
>
> For a mobile or desktop app there is no particular benefit in running a
> socket-based server, and it's much easier to install and start up an app
> that makes an in-process call to a dynamic library.
>
> So is this possible?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
>
--
Stig Bakken
CTO, Zedge.net - free your phone!
RE: Thrift as an in-process server?
Posted by David Bennett <da...@yorkage.com>.
Thanks for the response, but I'm not sure I understand.
By "server" I just mean the provider of the RPC service, and "client" is the consumer or caller of that service. The Thrift generated code is structured like that.
I'm asking about the possibility of building an application as a single process containing both Thrift client and server, to avoid the problems of starting up separate processes.
Yes, a buffer sounds good, but that still leaves out quite a few details, like how to get it across the language boundary.
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org
-----Original Message-----
From: Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: Tuesday, 15 September 2015 11:07 AM
To: user@thrift.apache.org
Subject: Re: Thrift as an in-process server?
I do not think you even need to server. You simply use thrift to write objects to a buffer/disk/ipc socket, and use thrift on the other side to read it back.
On Mon, Sep 14, 2015 at 8:03 PM, David Bennett <da...@yorkage.com> wrote:
> Thanks for your answer.
>
> Yes, I could see that it must be 'possible' from reading bits of the
> source code. What I can't tell is whether it's just a latent
> possibility lacking some key bits of code, or whether it's an actual
> possibility if I only knew how.
>
> Assume (a) a server in either Java or C#. Assume (b) a client in any
> of Java, C#, C++ or Ruby. How would you go about 'easily passed to the
> other side' for some combination of (a) and (b)?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
> -----Original Message-----
> From: Jens Geyer [mailto:jensgeyer@hotmail.com]
> Sent: Tuesday, 15 September 2015 7:20 AM
> To: user@thrift.apache.org
> Subject: Re: Thrift as an in-process server?
>
> > For a mobile or desktop app there is no particular benefit in
> > running a socket-based server
>
> Sure. And on top of it, you are by no means bound to sockets. The
> modular protocol/transport stack makes it possible, that's one of the
> really nice things about Thrift.
>
> For example, a stream transport is supported by a number of languages.
> It basically serializes and deserializes the data into and from a
> variable-sized buffer, not much magic behind. The buffer can be passed
> easily to the other side, and vice versa.
>
> Have fun,
> JensG
>
>
> -----Ursprüngliche Nachricht-----
> From: David Bennett
> Sent: Monday, September 14, 2015 3:57 AM
> To: user@thrift.apache.org
> Subject: Thrift as an in-process server?
>
> Is it possible to use Thrift for in-process cross-language marshalling?
>
> For a mobile or desktop app there is no particular benefit in running
> a socket-based server, and it's much easier to install and start up an
> app that makes an in-process call to a dynamic library.
>
> So is this possible?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
>
Re: Thrift as an in-process server?
Posted by Edward Capriolo <ed...@gmail.com>.
I do not think you even need to server. You simply use thrift to write
objects to a buffer/disk/ipc socket, and use thrift on the other side to
read it back.
On Mon, Sep 14, 2015 at 8:03 PM, David Bennett <da...@yorkage.com> wrote:
> Thanks for your answer.
>
> Yes, I could see that it must be 'possible' from reading bits of the
> source code. What I can't tell is whether it's just a latent possibility
> lacking some key bits of code, or whether it's an actual possibility if I
> only knew how.
>
> Assume (a) a server in either Java or C#. Assume (b) a client in any of
> Java, C#, C++ or Ruby. How would you go about 'easily passed to the other
> side' for some combination of (a) and (b)?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
> -----Original Message-----
> From: Jens Geyer [mailto:jensgeyer@hotmail.com]
> Sent: Tuesday, 15 September 2015 7:20 AM
> To: user@thrift.apache.org
> Subject: Re: Thrift as an in-process server?
>
> > For a mobile or desktop app there is no particular benefit in running
> > a socket-based server
>
> Sure. And on top of it, you are by no means bound to sockets. The modular
> protocol/transport stack makes it possible, that's one of the really nice
> things about Thrift.
>
> For example, a stream transport is supported by a number of languages. It
> basically serializes and deserializes the data into and from a
> variable-sized buffer, not much magic behind. The buffer can be passed
> easily to the other side, and vice versa.
>
> Have fun,
> JensG
>
>
> -----Ursprüngliche Nachricht-----
> From: David Bennett
> Sent: Monday, September 14, 2015 3:57 AM
> To: user@thrift.apache.org
> Subject: Thrift as an in-process server?
>
> Is it possible to use Thrift for in-process cross-language marshalling?
>
> For a mobile or desktop app there is no particular benefit in running a
> socket-based server, and it's much easier to install and start up an app
> that makes an in-process call to a dynamic library.
>
> So is this possible?
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
>
>
RE: Thrift as an in-process server?
Posted by David Bennett <da...@yorkage.com>.
Thanks for your answer.
Yes, I could see that it must be 'possible' from reading bits of the source code. What I can't tell is whether it's just a latent possibility lacking some key bits of code, or whether it's an actual possibility if I only knew how.
Assume (a) a server in either Java or C#. Assume (b) a client in any of Java, C#, C++ or Ruby. How would you go about 'easily passed to the other side' for some combination of (a) and (b)?
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org
-----Original Message-----
From: Jens Geyer [mailto:jensgeyer@hotmail.com]
Sent: Tuesday, 15 September 2015 7:20 AM
To: user@thrift.apache.org
Subject: Re: Thrift as an in-process server?
> For a mobile or desktop app there is no particular benefit in running
> a socket-based server
Sure. And on top of it, you are by no means bound to sockets. The modular protocol/transport stack makes it possible, that's one of the really nice things about Thrift.
For example, a stream transport is supported by a number of languages. It basically serializes and deserializes the data into and from a variable-sized buffer, not much magic behind. The buffer can be passed easily to the other side, and vice versa.
Have fun,
JensG
-----Ursprüngliche Nachricht-----
From: David Bennett
Sent: Monday, September 14, 2015 3:57 AM
To: user@thrift.apache.org
Subject: Thrift as an in-process server?
Is it possible to use Thrift for in-process cross-language marshalling?
For a mobile or desktop app there is no particular benefit in running a socket-based server, and it's much easier to install and start up an app that makes an in-process call to a dynamic library.
So is this possible?
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org
Re: Thrift as an in-process server?
Posted by Jens Geyer <je...@hotmail.com>.
> For a mobile or desktop app there is no particular benefit in running a
> socket-based server
Sure. And on top of it, you are by no means bound to sockets. The modular
protocol/transport stack makes it possible, that's one of the really nice
things about Thrift.
For example, a stream transport is supported by a number of languages. It
basically serializes and deserializes the data into and from a
variable-sized buffer, not much magic behind. The buffer can be passed
easily to the other side, and vice versa.
Have fun,
JensG
-----Ursprüngliche Nachricht-----
From: David Bennett
Sent: Monday, September 14, 2015 3:57 AM
To: user@thrift.apache.org
Subject: Thrift as an in-process server?
Is it possible to use Thrift for in-process cross-language marshalling?
For a mobile or desktop app there is no particular benefit in running a
socket-based server, and it's much easier to install and start up an app
that makes an in-process call to a dynamic library.
So is this possible?
Regards
David M Bennett FACS
Andl - A New Database Language - andl.org