You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@plc4x.apache.org by Christofer Dutz <ch...@c-ware.de> on 2020/04/03 14:04:23 UTC
[DISCUSS] Alternatives to Thrift?
Hi all,
as in a few weeks my PLC4C project will hopefully entering a phase where I have to implement the “proxy” functionality, I would like to start early and discuss with you options instead of Thrift.
Why not use Thrift?
Well I will be targeting non POSIX systems with PLC4C. Unfortunately Thrift does have a C-GLIB client, however this code seems to utilize Boost and Boost relies on POSSIX systems so I couldn’t use it in my initiative. So I need something else … something lightweight but also something you can secure with TLS.
Julian suggested
gRPC
It looks very promising … even if there are no native C full implementations available out there. It seems to be based on Protocol Buffers from Google. There are implementations available in C for that. gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use Protocol Buffers directly? I mean we have both parts of the equation in our hands. gRPC would allow using our PLC4X proxy transport with any other gPRC client, but then we would probably lose our ability to just play around with the format. Also do I like the idea of making people use PLC4X on both sides and not just pick some parts out.
What do you think? Any other transports out there that would allow secure communication between two PLC4X nodes?
When doing research before starting the whole mspec thing I also had a look at Apache Avro
Some more links:
* https://capnproto.org/ (Authored by Kenton Varda, the primary author of Protocol Buffers version 2)
* https://google.github.io/flatbuffers/
* https://msgpack.org/
* …
Chris
Re: [DISCUSS] Alternatives to Thrift?
Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hey,
I used avro several times and the difference is that its dynamic, so no definition of objects at compile time but rather at runtime.
Julian
Am 03.04.20, 19:28 schrieb "Christofer Dutz" <ch...@c-ware.de>:
In this usecase, we could simply open a socket, secure that via tls and then just pass serialized objects over that connection. That's why I think it's usable.
Http2 sounds like adding a Webserver to the equation. I wouldn't like to do that on a system with a few hundred k memory.
Chris
________________________________
Von: Lukas Ott <ot...@gmail.com>
Gesendet: Freitag, 3. April 2020 16:35
An: dev@plc4x.apache.org <de...@plc4x.apache.org>
Betreff: Re: [DISCUSS] Alternatives to Thrift?
Hi,
+1 for Julian s suggestion to use gRPC. That is one of the most promising
approach, if someone else has not other suggestion.
Question: How does Apache Avro fit into that picture? (vs Parquet, Apache
ORC) - > Maybe check also Apache Arrow as they aim for cross-language ?
Lukas
Am Fr., 3. Apr. 2020 um 16:04 Uhr schrieb Christofer Dutz <
christofer.dutz@c-ware.de>:
> Hi all,
>
> as in a few weeks my PLC4C project will hopefully entering a phase where I
> have to implement the “proxy” functionality, I would like to start early
> and discuss with you options instead of Thrift.
>
> Why not use Thrift?
> Well I will be targeting non POSIX systems with PLC4C. Unfortunately
> Thrift does have a C-GLIB client, however this code seems to utilize Boost
> and Boost relies on POSSIX systems so I couldn’t use it in my initiative.
> So I need something else … something lightweight but also something you can
> secure with TLS.
>
> Julian suggested
>
> gRPC
>
> It looks very promising … even if there are no native C full
> implementations available out there. It seems to be based on Protocol
> Buffers from Google. There are implementations available in C for that.
> gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use
> Protocol Buffers directly? I mean we have both parts of the equation in our
> hands. gRPC would allow using our PLC4X proxy transport with any other gPRC
> client, but then we would probably lose our ability to just play around
> with the format. Also do I like the idea of making people use PLC4X on both
> sides and not just pick some parts out.
>
> What do you think? Any other transports out there that would allow secure
> communication between two PLC4X nodes?
>
> When doing research before starting the whole mspec thing I also had a
> look at Apache Avro
>
> Some more links:
>
> * https://capnproto.org/ (Authored by Kenton Varda, the primary
> author of Protocol Buffers version 2)
> * https://google.github.io/flatbuffers/
> * https://msgpack.org/
> * …
>
> Chris
>
Re: [DISCUSS] Alternatives to Thrift?
Posted by Christofer Dutz <ch...@c-ware.de>.
In this usecase, we could simply open a socket, secure that via tls and then just pass serialized objects over that connection. That's why I think it's usable.
Http2 sounds like adding a Webserver to the equation. I wouldn't like to do that on a system with a few hundred k memory.
Chris
________________________________
Von: Lukas Ott <ot...@gmail.com>
Gesendet: Freitag, 3. April 2020 16:35
An: dev@plc4x.apache.org <de...@plc4x.apache.org>
Betreff: Re: [DISCUSS] Alternatives to Thrift?
Hi,
+1 for Julian s suggestion to use gRPC. That is one of the most promising
approach, if someone else has not other suggestion.
Question: How does Apache Avro fit into that picture? (vs Parquet, Apache
ORC) - > Maybe check also Apache Arrow as they aim for cross-language ?
Lukas
Am Fr., 3. Apr. 2020 um 16:04 Uhr schrieb Christofer Dutz <
christofer.dutz@c-ware.de>:
> Hi all,
>
> as in a few weeks my PLC4C project will hopefully entering a phase where I
> have to implement the “proxy” functionality, I would like to start early
> and discuss with you options instead of Thrift.
>
> Why not use Thrift?
> Well I will be targeting non POSIX systems with PLC4C. Unfortunately
> Thrift does have a C-GLIB client, however this code seems to utilize Boost
> and Boost relies on POSSIX systems so I couldn’t use it in my initiative.
> So I need something else … something lightweight but also something you can
> secure with TLS.
>
> Julian suggested
>
> gRPC
>
> It looks very promising … even if there are no native C full
> implementations available out there. It seems to be based on Protocol
> Buffers from Google. There are implementations available in C for that.
> gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use
> Protocol Buffers directly? I mean we have both parts of the equation in our
> hands. gRPC would allow using our PLC4X proxy transport with any other gPRC
> client, but then we would probably lose our ability to just play around
> with the format. Also do I like the idea of making people use PLC4X on both
> sides and not just pick some parts out.
>
> What do you think? Any other transports out there that would allow secure
> communication between two PLC4X nodes?
>
> When doing research before starting the whole mspec thing I also had a
> look at Apache Avro
>
> Some more links:
>
> * https://capnproto.org/ (Authored by Kenton Varda, the primary
> author of Protocol Buffers version 2)
> * https://google.github.io/flatbuffers/
> * https://msgpack.org/
> * …
>
> Chris
>
Re: [DISCUSS] Alternatives to Thrift?
Posted by Lukas Ott <ot...@gmail.com>.
Hi,
+1 for Julian s suggestion to use gRPC. That is one of the most promising
approach, if someone else has not other suggestion.
Question: How does Apache Avro fit into that picture? (vs Parquet, Apache
ORC) - > Maybe check also Apache Arrow as they aim for cross-language ?
Lukas
Am Fr., 3. Apr. 2020 um 16:04 Uhr schrieb Christofer Dutz <
christofer.dutz@c-ware.de>:
> Hi all,
>
> as in a few weeks my PLC4C project will hopefully entering a phase where I
> have to implement the “proxy” functionality, I would like to start early
> and discuss with you options instead of Thrift.
>
> Why not use Thrift?
> Well I will be targeting non POSIX systems with PLC4C. Unfortunately
> Thrift does have a C-GLIB client, however this code seems to utilize Boost
> and Boost relies on POSSIX systems so I couldn’t use it in my initiative.
> So I need something else … something lightweight but also something you can
> secure with TLS.
>
> Julian suggested
>
> gRPC
>
> It looks very promising … even if there are no native C full
> implementations available out there. It seems to be based on Protocol
> Buffers from Google. There are implementations available in C for that.
> gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use
> Protocol Buffers directly? I mean we have both parts of the equation in our
> hands. gRPC would allow using our PLC4X proxy transport with any other gPRC
> client, but then we would probably lose our ability to just play around
> with the format. Also do I like the idea of making people use PLC4X on both
> sides and not just pick some parts out.
>
> What do you think? Any other transports out there that would allow secure
> communication between two PLC4X nodes?
>
> When doing research before starting the whole mspec thing I also had a
> look at Apache Avro
>
> Some more links:
>
> * https://capnproto.org/ (Authored by Kenton Varda, the primary
> author of Protocol Buffers version 2)
> * https://google.github.io/flatbuffers/
> * https://msgpack.org/
> * …
>
> Chris
>
Re: [DISCUSS] Alternatives to Thrift?
Posted by Christofer Dutz <ch...@c-ware.de>.
And what would you guys think of an mspec version of gRPC ...
so we wouldn't be defining a new protocol, but we wouldn't be bringing in the weight of a full webserver and full implementation of all bells and whistles?
Chris
Am 07.04.20, 10:18 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:
Agree. I would prefer gRPC as its "richer" than MQTT as MQTT is only transport and gRPC is transport + data description.
J
Am 07.04.20, 10:01 schrieb "Strljic, Matthias Milan" <ma...@isw.uni-stuttgart.de>:
Hi Chris,
if gRPC could be a match i would go first with that. I would not go the direction of implementing a complete new protocol. Rather use gRPC or MQTT.
Greetings
Mathi
________________________________
Von: Christofer Dutz <ch...@c-ware.de>
Gesendet: Freitag, 3. April 2020 16:04:23
An: dev@plc4x.apache.org
Betreff: [DISCUSS] Alternatives to Thrift?
Hi all,
as in a few weeks my PLC4C project will hopefully entering a phase where I have to implement the “proxy” functionality, I would like to start early and discuss with you options instead of Thrift.
Why not use Thrift?
Well I will be targeting non POSIX systems with PLC4C. Unfortunately Thrift does have a C-GLIB client, however this code seems to utilize Boost and Boost relies on POSSIX systems so I couldn’t use it in my initiative. So I need something else … something lightweight but also something you can secure with TLS.
Julian suggested
gRPC
It looks very promising … even if there are no native C full implementations available out there. It seems to be based on Protocol Buffers from Google. There are implementations available in C for that. gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use Protocol Buffers directly? I mean we have both parts of the equation in our hands. gRPC would allow using our PLC4X proxy transport with any other gPRC client, but then we would probably lose our ability to just play around with the format. Also do I like the idea of making people use PLC4X on both sides and not just pick some parts out.
What do you think? Any other transports out there that would allow secure communication between two PLC4X nodes?
When doing research before starting the whole mspec thing I also had a look at Apache Avro
Some more links:
* https://capnproto.org/ (Authored by Kenton Varda, the primary author of Protocol Buffers version 2)
* https://google.github.io/flatbuffers/
* https://msgpack.org/
* …
Chris
Re: [DISCUSS] Alternatives to Thrift?
Posted by Julian Feinauer <j....@pragmaticminds.de>.
Agree. I would prefer gRPC as its "richer" than MQTT as MQTT is only transport and gRPC is transport + data description.
J
Am 07.04.20, 10:01 schrieb "Strljic, Matthias Milan" <ma...@isw.uni-stuttgart.de>:
Hi Chris,
if gRPC could be a match i would go first with that. I would not go the direction of implementing a complete new protocol. Rather use gRPC or MQTT.
Greetings
Mathi
________________________________
Von: Christofer Dutz <ch...@c-ware.de>
Gesendet: Freitag, 3. April 2020 16:04:23
An: dev@plc4x.apache.org
Betreff: [DISCUSS] Alternatives to Thrift?
Hi all,
as in a few weeks my PLC4C project will hopefully entering a phase where I have to implement the “proxy” functionality, I would like to start early and discuss with you options instead of Thrift.
Why not use Thrift?
Well I will be targeting non POSIX systems with PLC4C. Unfortunately Thrift does have a C-GLIB client, however this code seems to utilize Boost and Boost relies on POSSIX systems so I couldn’t use it in my initiative. So I need something else … something lightweight but also something you can secure with TLS.
Julian suggested
gRPC
It looks very promising … even if there are no native C full implementations available out there. It seems to be based on Protocol Buffers from Google. There are implementations available in C for that. gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use Protocol Buffers directly? I mean we have both parts of the equation in our hands. gRPC would allow using our PLC4X proxy transport with any other gPRC client, but then we would probably lose our ability to just play around with the format. Also do I like the idea of making people use PLC4X on both sides and not just pick some parts out.
What do you think? Any other transports out there that would allow secure communication between two PLC4X nodes?
When doing research before starting the whole mspec thing I also had a look at Apache Avro
Some more links:
* https://capnproto.org/ (Authored by Kenton Varda, the primary author of Protocol Buffers version 2)
* https://google.github.io/flatbuffers/
* https://msgpack.org/
* …
Chris
AW: [DISCUSS] Alternatives to Thrift?
Posted by "Strljic, Matthias Milan" <ma...@isw.uni-stuttgart.de>.
Hi Chris,
if gRPC could be a match i would go first with that. I would not go the direction of implementing a complete new protocol. Rather use gRPC or MQTT.
Greetings
Mathi
________________________________
Von: Christofer Dutz <ch...@c-ware.de>
Gesendet: Freitag, 3. April 2020 16:04:23
An: dev@plc4x.apache.org
Betreff: [DISCUSS] Alternatives to Thrift?
Hi all,
as in a few weeks my PLC4C project will hopefully entering a phase where I have to implement the “proxy” functionality, I would like to start early and discuss with you options instead of Thrift.
Why not use Thrift?
Well I will be targeting non POSIX systems with PLC4C. Unfortunately Thrift does have a C-GLIB client, however this code seems to utilize Boost and Boost relies on POSSIX systems so I couldn’t use it in my initiative. So I need something else … something lightweight but also something you can secure with TLS.
Julian suggested
gRPC
It looks very promising … even if there are no native C full implementations available out there. It seems to be based on Protocol Buffers from Google. There are implementations available in C for that. gRPC seems to run over HTTP2/TCP … not quite sure if we shouldn’t just use Protocol Buffers directly? I mean we have both parts of the equation in our hands. gRPC would allow using our PLC4X proxy transport with any other gPRC client, but then we would probably lose our ability to just play around with the format. Also do I like the idea of making people use PLC4X on both sides and not just pick some parts out.
What do you think? Any other transports out there that would allow secure communication between two PLC4X nodes?
When doing research before starting the whole mspec thing I also had a look at Apache Avro
Some more links:
* https://capnproto.org/ (Authored by Kenton Varda, the primary author of Protocol Buffers version 2)
* https://google.github.io/flatbuffers/
* https://msgpack.org/
* …
Chris