You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Rush Manbert <ru...@manbert.com> on 2008/05/13 20:37:54 UTC
"Deadly Embrace" between included thrift source files
In general, I'm trying to define the following in my thrift source file:
struct foo
{
1:list<bar> ref,
}
struct bar
{
1:list<foo> ref,
}
but the compiler chokes on this, telling me "Type "bar" has not been
defined." at line 3.
For us, it gets worse than that because we want to autogenerate our
thrift sources from a data model, and we would generate this:
File foo.thrift:
---------------
include "bar.thrift"
struct foo
{
1:list<bar> ref,
}
File bar.thrift:
---------------
include "foo.thrift"
struct bar
{
1:list<foo> ref,
}
Of course, the real case is much more complicated, but this
illustrates the problem. If we compile foo.thrift, the compiler just
spins forever between the two thrift files.
Is there some way to handle these cases currently? (I would think that
Facebook would have run into this problem internally before now and
would have had to solve it.)
If not, are there plans to add either forward struct declarations, or
an "include once" mechanism, or both to the compiler?
Thanks,
Rush
Re: "Deadly Embrace" between included thrift source files
Posted by Rush Manbert <ru...@manbert.com>.
Well, it was worth a try. Our code gen hack seems to work, so we'll
continue with that. (No compiler guys here, so this seems safer for us.)
- Rush
On May 13, 2008, at 3:29 PM, David Reiss wrote:
> I understand the use case, but there are still no plans to support
> this.
> If someone submits a working patch, we will, of course, consider
> including it.
>
> --David
>
> Rush Manbert wrote:
>> A very common use case (ours, in fact) is a top level object that has
>> a list of child objects of the same type.
>>
>> struct myObj {
>> 1: i32 something,
>> 2: list<myObj> children,
>> }
>>
>> This occurs quite naturally in a database representation of complex
>> data structures. In our case we are querying for the top level
>> object,
>> which also gets us the list of child objects. We want to marshall the
>> whole thing into a thrift structure on the server side and hand it
>> out
>> to our clients.
>>
>> Obviously the underlying languages support this. We think we can hack
>> the thrift source code we generate to make it happen, but it's a hack
>> that depends on the way code is generated today, and it would
>> certainly be a lot cleaner if the compiler just supported it.
>>
>> - Rush
>>
>> On May 13, 2008, at 12:26 PM, David Reiss wrote:
>>
>>> This is definitely a bug we should fix. However, there are no
>>> current
>>> plans to allow for recursive types, so I don't think there is a use
>>> case
>>> for doing recursive includes.
>>>
>>> --David
>>>
>>> Rush Manbert wrote:
>>>> In general, I'm trying to define the following in my thrift source
>>>> file:
>>>> struct foo
>>>> {
>>>> 1:list<bar> ref,
>>>> }
>>>>
>>>> struct bar
>>>> {
>>>> 1:list<foo> ref,
>>>> }
>>>>
>>>> but the compiler chokes on this, telling me "Type "bar" has not
>>>> been
>>>> defined." at line 3.
>>>>
>>>> For us, it gets worse than that because we want to autogenerate our
>>>> thrift sources from a data model, and we would generate this:
>>>>
>>>> File foo.thrift:
>>>> ---------------
>>>> include "bar.thrift"
>>>>
>>>> struct foo
>>>> {
>>>> 1:list<bar> ref,
>>>> }
>>>>
>>>> File bar.thrift:
>>>> ---------------
>>>> include "foo.thrift"
>>>>
>>>> struct bar
>>>> {
>>>> 1:list<foo> ref,
>>>> }
>>>>
>>>> Of course, the real case is much more complicated, but this
>>>> illustrates the problem. If we compile foo.thrift, the compiler
>>>> just
>>>> spins forever between the two thrift files.
>>>>
>>>> Is there some way to handle these cases currently? (I would think
>>>> that
>>>> Facebook would have run into this problem internally before now and
>>>> would have had to solve it.)
>>>>
>>>> If not, are there plans to add either forward struct
>>>> declarations, or
>>>> an "include once" mechanism, or both to the compiler?
>>>>
>>>> Thanks,
>>>> Rush
>>>>
>>
Re: "Deadly Embrace" between included thrift source files
Posted by David Reiss <dr...@facebook.com>.
I understand the use case, but there are still no plans to support this.
If someone submits a working patch, we will, of course, consider
including it.
--David
Rush Manbert wrote:
> A very common use case (ours, in fact) is a top level object that has
> a list of child objects of the same type.
>
> struct myObj {
> 1: i32 something,
> 2: list<myObj> children,
> }
>
> This occurs quite naturally in a database representation of complex
> data structures. In our case we are querying for the top level object,
> which also gets us the list of child objects. We want to marshall the
> whole thing into a thrift structure on the server side and hand it out
> to our clients.
>
> Obviously the underlying languages support this. We think we can hack
> the thrift source code we generate to make it happen, but it's a hack
> that depends on the way code is generated today, and it would
> certainly be a lot cleaner if the compiler just supported it.
>
> - Rush
>
> On May 13, 2008, at 12:26 PM, David Reiss wrote:
>
>> This is definitely a bug we should fix. However, there are no current
>> plans to allow for recursive types, so I don't think there is a use
>> case
>> for doing recursive includes.
>>
>> --David
>>
>> Rush Manbert wrote:
>>> In general, I'm trying to define the following in my thrift source
>>> file:
>>> struct foo
>>> {
>>> 1:list<bar> ref,
>>> }
>>>
>>> struct bar
>>> {
>>> 1:list<foo> ref,
>>> }
>>>
>>> but the compiler chokes on this, telling me "Type "bar" has not been
>>> defined." at line 3.
>>>
>>> For us, it gets worse than that because we want to autogenerate our
>>> thrift sources from a data model, and we would generate this:
>>>
>>> File foo.thrift:
>>> ---------------
>>> include "bar.thrift"
>>>
>>> struct foo
>>> {
>>> 1:list<bar> ref,
>>> }
>>>
>>> File bar.thrift:
>>> ---------------
>>> include "foo.thrift"
>>>
>>> struct bar
>>> {
>>> 1:list<foo> ref,
>>> }
>>>
>>> Of course, the real case is much more complicated, but this
>>> illustrates the problem. If we compile foo.thrift, the compiler just
>>> spins forever between the two thrift files.
>>>
>>> Is there some way to handle these cases currently? (I would think
>>> that
>>> Facebook would have run into this problem internally before now and
>>> would have had to solve it.)
>>>
>>> If not, are there plans to add either forward struct declarations, or
>>> an "include once" mechanism, or both to the compiler?
>>>
>>> Thanks,
>>> Rush
>>>
>
Re: "Deadly Embrace" between included thrift source files
Posted by Rush Manbert <ru...@manbert.com>.
A very common use case (ours, in fact) is a top level object that has
a list of child objects of the same type.
struct myObj {
1: i32 something,
2: list<myObj> children,
}
This occurs quite naturally in a database representation of complex
data structures. In our case we are querying for the top level object,
which also gets us the list of child objects. We want to marshall the
whole thing into a thrift structure on the server side and hand it out
to our clients.
Obviously the underlying languages support this. We think we can hack
the thrift source code we generate to make it happen, but it's a hack
that depends on the way code is generated today, and it would
certainly be a lot cleaner if the compiler just supported it.
- Rush
On May 13, 2008, at 12:26 PM, David Reiss wrote:
> This is definitely a bug we should fix. However, there are no current
> plans to allow for recursive types, so I don't think there is a use
> case
> for doing recursive includes.
>
> --David
>
> Rush Manbert wrote:
>> In general, I'm trying to define the following in my thrift source
>> file:
>> struct foo
>> {
>> 1:list<bar> ref,
>> }
>>
>> struct bar
>> {
>> 1:list<foo> ref,
>> }
>>
>> but the compiler chokes on this, telling me "Type "bar" has not been
>> defined." at line 3.
>>
>> For us, it gets worse than that because we want to autogenerate our
>> thrift sources from a data model, and we would generate this:
>>
>> File foo.thrift:
>> ---------------
>> include "bar.thrift"
>>
>> struct foo
>> {
>> 1:list<bar> ref,
>> }
>>
>> File bar.thrift:
>> ---------------
>> include "foo.thrift"
>>
>> struct bar
>> {
>> 1:list<foo> ref,
>> }
>>
>> Of course, the real case is much more complicated, but this
>> illustrates the problem. If we compile foo.thrift, the compiler just
>> spins forever between the two thrift files.
>>
>> Is there some way to handle these cases currently? (I would think
>> that
>> Facebook would have run into this problem internally before now and
>> would have had to solve it.)
>>
>> If not, are there plans to add either forward struct declarations, or
>> an "include once" mechanism, or both to the compiler?
>>
>> Thanks,
>> Rush
>>
Re: "Deadly Embrace" between included thrift source files
Posted by David Reiss <dr...@facebook.com>.
This is definitely a bug we should fix. However, there are no current
plans to allow for recursive types, so I don't think there is a use case
for doing recursive includes.
--David
Rush Manbert wrote:
> In general, I'm trying to define the following in my thrift source file:
> struct foo
> {
> 1:list<bar> ref,
> }
>
> struct bar
> {
> 1:list<foo> ref,
> }
>
> but the compiler chokes on this, telling me "Type "bar" has not been
> defined." at line 3.
>
> For us, it gets worse than that because we want to autogenerate our
> thrift sources from a data model, and we would generate this:
>
> File foo.thrift:
> ---------------
> include "bar.thrift"
>
> struct foo
> {
> 1:list<bar> ref,
> }
>
> File bar.thrift:
> ---------------
> include "foo.thrift"
>
> struct bar
> {
> 1:list<foo> ref,
> }
>
> Of course, the real case is much more complicated, but this
> illustrates the problem. If we compile foo.thrift, the compiler just
> spins forever between the two thrift files.
>
> Is there some way to handle these cases currently? (I would think that
> Facebook would have run into this problem internally before now and
> would have had to solve it.)
>
> If not, are there plans to add either forward struct declarations, or
> an "include once" mechanism, or both to the compiler?
>
> Thanks,
> Rush
>