You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Jens Geyer <je...@hotmail.com> on 2013/04/13 14:22:07 UTC
Recursion in structures
Hi *,
I know, I read it somewhere but I couldn't find it again, neither in the
code, the whitepaper nor in the Wiki. But somewhere there is a remark or
comment about why recursive types are not supported in Thrift. However, we
frequently run into a situation where we experience the need for such
structures, e.g. with hierarchical data:
struct treenode
{
1: string name
2: list<treenode> childs
}
Similarly, one could not do things like the following with Thrift:
struct foo
{
1: bar somefield
}
struct bar
{
1: foo anotherfield
}
I would really like to see Thrift enhanced here. Especially transporting
hierachical data through a Thrift-based API causes a lot of extra headaches.
So here are my questions:
(1) Could anyone provide a pointer to the comment or remark I mentioned
above?
(2) What could prevent us from enhancing the IDL this way? Particularly, do
you see any obious (or not-so-obvious) major problems here with the parser
and the generators?
(3) Do all supported languages allow such constructs?
Thanks in advance for any input you might have,
JensG
Re: Recursion in structures
Posted by Jens Geyer <je...@hotmail.com>.
Thanks Bill,
based on your answer I searched the mailing list archives and found a number
of discussions. Very helpful.
Interestingly, most of the use cases that pop up every now and then seem
indeed to deal with data organized in trees ... looks as if we really should
think about it. From what I read so far, improvements in that area could add
real value to Thrift.
BTW, the comment I was looking for is here:
http://markmail.org/thread/4pwhw5d254zrphv5
Regards,
JensG
-----Ursprüngliche Nachricht-----
From: Bill Dortch
Sent: Saturday, April 13, 2013 4:08 PM
To: dev@thrift.apache.org
Subject: Re: Recursion in structures
Hi Jens,
I raised this a couple of years ago in an off-list conversation, and the
answer I got was basically that it would be dangerous because Thrift does
not have a mechanism for detecting or coping with cycles in graphs of
objects. That's true, and that would be non-trivial to implement. However,
there are many use cases for trees or other directed acyclic graphs that
could benefit from this capability, and in which a cycle would be a
programmer error, not something Thrift needed to handle.
BTW, I ended up not using Thrift, mostly for this reason, but I did write a
JavaCC-based Thrift parser (called Horatio) that does handle forward
references, and use it now to generate Thrift-ish objects, though with a
lighter-weight protocol/transport framework, and only for Java and
Javascript objects (though the generator is template-based, so you could
easily generate anything from the model, including proper Thrift objects).
I've been meaning to post it on GitHub at some point -- let me know if
you're interested in taking a look at it.
Cheers,
Bill Dortch
On Sat, Apr 13, 2013 at 5:22 AM, Jens Geyer <je...@hotmail.com> wrote:
> Hi *,
>
> I know, I read it somewhere but I couldn't find it again, neither in the
> code, the whitepaper nor in the Wiki. But somewhere there is a remark or
> comment about why recursive types are not supported in Thrift. However, we
> frequently run into a situation where we experience the need for such
> structures, e.g. with hierarchical data:
>
> struct treenode
> {
> 1: string name
> 2: list<treenode> childs
> }
>
> Similarly, one could not do things like the following with Thrift:
>
> struct foo
> {
> 1: bar somefield
> }
>
> struct bar
> {
> 1: foo anotherfield
> }
>
> I would really like to see Thrift enhanced here. Especially transporting
> hierachical data through a Thrift-based API causes a lot of extra
> headaches. So here are my questions:
>
> (1) Could anyone provide a pointer to the comment or remark I mentioned
> above?
>
> (2) What could prevent us from enhancing the IDL this way? Particularly,
> do you see any obious (or not-so-obvious) major problems here with the
> parser and the generators?
>
> (3) Do all supported languages allow such constructs?
>
>
> Thanks in advance for any input you might have,
> JensG
>
>
Re: Recursion in structures
Posted by Bill Dortch <bi...@gmail.com>.
Hi Jens,
I raised this a couple of years ago in an off-list conversation, and the
answer I got was basically that it would be dangerous because Thrift does
not have a mechanism for detecting or coping with cycles in graphs of
objects. That's true, and that would be non-trivial to implement. However,
there are many use cases for trees or other directed acyclic graphs that
could benefit from this capability, and in which a cycle would be a
programmer error, not something Thrift needed to handle.
BTW, I ended up not using Thrift, mostly for this reason, but I did write a
JavaCC-based Thrift parser (called Horatio) that does handle forward
references, and use it now to generate Thrift-ish objects, though with a
lighter-weight protocol/transport framework, and only for Java and
Javascript objects (though the generator is template-based, so you could
easily generate anything from the model, including proper Thrift objects).
I've been meaning to post it on GitHub at some point -- let me know if
you're interested in taking a look at it.
Cheers,
Bill Dortch
On Sat, Apr 13, 2013 at 5:22 AM, Jens Geyer <je...@hotmail.com> wrote:
> Hi *,
>
> I know, I read it somewhere but I couldn't find it again, neither in the
> code, the whitepaper nor in the Wiki. But somewhere there is a remark or
> comment about why recursive types are not supported in Thrift. However, we
> frequently run into a situation where we experience the need for such
> structures, e.g. with hierarchical data:
>
> struct treenode
> {
> 1: string name
> 2: list<treenode> childs
> }
>
> Similarly, one could not do things like the following with Thrift:
>
> struct foo
> {
> 1: bar somefield
> }
>
> struct bar
> {
> 1: foo anotherfield
> }
>
> I would really like to see Thrift enhanced here. Especially transporting
> hierachical data through a Thrift-based API causes a lot of extra
> headaches. So here are my questions:
>
> (1) Could anyone provide a pointer to the comment or remark I mentioned
> above?
>
> (2) What could prevent us from enhancing the IDL this way? Particularly,
> do you see any obious (or not-so-obvious) major problems here with the
> parser and the generators?
>
> (3) Do all supported languages allow such constructs?
>
>
> Thanks in advance for any input you might have,
> JensG
>
>
Re: Recursion in structures
Posted by Chet Murthy <mu...@gmail.com>.
I'm not a longtime user or developer of Thrift, but I've worked with a
lot of RPC systems in the past, and have implemented a CORBA ORB in a
product.
When Thrift came out, I saw two excellent things in it:
(1) explicit support and facilitation for versioning, both of
interfaces and of the framework itself. But esp. of interfaces.
(2) A decision to avoid "local/remote transparency". This shows up in
lots of ways, but one way is that the only data-structures that are
supported are trees. No DAGs, no graphs.
Having implemented an ORB, I can attest to the difficulties of
supporting these things in an RPC system. Specifically, you have to
figure out where DAGness can occur, and use hashtables to keep track
of pointers at those spots. It's a pain, and furthermore, it's
inefficient.
I think the Thrift decision to privilege "flat data" was a good one.
Sure, it isn't ideal for all purposes, but it's better to force
applications that need more, to code it themselves, and keep things
simple and fast for the bulk of apps.
At least, that's my 2c.
--chet--
On Saturday, April 13, 2013 02:22:07 PM Jens Geyer wrote:
> Hi *,
>
> I know, I read it somewhere but I couldn't find it again, neither in the
> code, the whitepaper nor in the Wiki. But somewhere there is a remark or
> comment about why recursive types are not supported in Thrift. However, we
> frequently run into a situation where we experience the need for such
> structures, e.g. with hierarchical data:
>
> struct treenode
> {
> 1: string name
> 2: list<treenode> childs
> }
>
> Similarly, one could not do things like the following with Thrift:
>
> struct foo
> {
> 1: bar somefield
> }
>
> struct bar
> {
> 1: foo anotherfield
> }
>
> I would really like to see Thrift enhanced here. Especially transporting
> hierachical data through a Thrift-based API causes a lot of extra headaches.
> So here are my questions:
>
> (1) Could anyone provide a pointer to the comment or remark I mentioned
> above?
>
> (2) What could prevent us from enhancing the IDL this way? Particularly, do
> you see any obious (or not-so-obvious) major problems here with the parser
> and the generators?
>
> (3) Do all supported languages allow such constructs?
>
>
> Thanks in advance for any input you might have,
> JensG