You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by Bryan Duxbury <br...@rapleaf.com> on 2009/03/05 22:34:23 UTC
Some thoughts about changes to Thrift
Hey guys,
I wanted to run an idea past everyone. There was an email to the list
a few days ago about Ruby enum validation of implicit values that got
me thinking that Thrift allows a lot of design decisions that aren't
really in line with best practices. In order to achieve respectable
forward and backward compatibility, one of Thrift's main stated
goals, there are a bunch of things you *should* do, but that aren't
enforced. For instance, implicit enum values and optional field ids
in structs and on service method parameters. These are things that a
careful Thrift user should never really do, but since they aren't
enforced, there's risk for new users to do the wrong thing.
My idea is that we should change the Thrift IDL essentially to force
these best practices on users. The obvious benefit is that even a new
user will be pushed to do the right thing, avoiding potential
pitfalls in writing their definitions. The downside is that .thrift
files will be a little more verbose on average. (It's also a non-
backwards-compatible change, so everyone will have to update their
Thrift definitions.) At least in my mind, it seems like the positives
would outweigh the benefits, and it would free us up to do things
like get rid of autogenerated negative field ids.
What are everyone's thoughts on this idea?
-Bryan
Re: Some thoughts about changes to Thrift
Posted by Michael Greene <mi...@gmail.com>.
On Fri, Mar 6, 2009 at 11:26 AM, Andrew McGeachie <ge...@gmail.com> wrote:
> +1
>
> - a
>
> On Mar 6, 2009, at 11:02 AM, Bryan Duxbury wrote:
>
>> It seems like there's some consensus on a JavaCC based compiler. Should we
>> open an JIRA issue and start to explore feasibility?
>> -Bryan
I'm not at all opposed to this idea, as I'd like to see an alternative
compiler developed. I definitely think people should explore this
option. However, I don't think there's consensus at all that it
should be JavaCC-based, and durin the feasibility stage should
continue to explore alternatives.
What are the requirements, as I see them?
Supports LALR or better, for some meaning of better
Easily portable to Linux, Windows, Solaris, OSX and the BSDs
Doesn't require core packages [gcc, jdk, ruby, python, c runtime]
released after a certain date (late 2004 perhaps)
I would be in favor of any toolset meeting these requirements over the
existing one, but there are many options out there. If JavaCC is what
gets it done, peachy.
Michael
Re: Some thoughts about changes to Thrift
Posted by Andrew McGeachie <ge...@gmail.com>.
+1
- a
On Mar 6, 2009, at 11:02 AM, Bryan Duxbury wrote:
> It seems like there's some consensus on a JavaCC based compiler.
> Should we open an JIRA issue and start to explore feasibility?
> -Bryan
>
> On Mar 6, 2009, at 3:03 AM, Esteve Fernandez wrote:
>
>> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>>> That's pretty much my fault.
>>
>> Actually the current Thrift compiler served its puporse quite
>> decently, but
>> the coming of new languages and features has made it (along with the
>> generators) a bit more complex and less easy to maintain.
>>
>>> I personally like Java and think it'd be a decent choice, but
>>> there do seem
>>> to be a decent number of people out there building services who
>>> seem to
>>> vehemently hate Java. Lots of *nix systems do not have a JVM or JDK
>>> installed by default -- and it's pretty annoying for users who
>>> aren't
>>> writing services in Java to have to install it to build the
>>> compiler.
>>
>> I'd argue for writing a parser in Java and use a template engine
>> (Velocity,
>> Freemarker, etc.). Why not Python? Dunno :-) I think Java is more
>> widely
>> deployed, it has better tools for scanning and parsing and more
>> developers.
>> Etch, another incubating project with similar goals to Thrift, uses
>> JavaCC
>> for parsing and Velocity as template engine for its compiler and
>> generators,
>> even though it also emits C#
>>
>> Using a template engine would make development of new features and
>> the
>> inclusion of more languages much easier. Instead of having to hack a
>> generator to add some new feature, we could simply supply a new
>> template
>> file. For example, at some point in the future I'd like to build a
>> generator
>> for Javascript, and if we used a template engine, it would be a
>> matter of
>> writing a minimal generator and a bunch of templates.
>>
>> Cheers.
>
Re: Some thoughts about changes to Thrift
Posted by Chris Lamprecht <cl...@gmail.com>.
You might also consider JFlex (http://jflex.de/) -- I haven't done much with
either JavaCC or JFlex other than modify existing grammars, but the Lucene
guys moved from JavaCC to JFlex for their tokenizer -
http://issues.apache.org/jira/browse/LUCENE-966. If I remember correctly,
two benefits were (1) faster tokenizing, and (2) simpler grammar definition.
Performance probably isn't an issue for Thrift, but a simpler grammar may
be. As I said, I can't give a good comparison but maybe someone else on the
mailing list can.
-chris
On Fri, Mar 6, 2009 at 10:02 AM, Bryan Duxbury <br...@rapleaf.com> wrote:
> It seems like there's some consensus on a JavaCC based compiler. Should we
> open an JIRA issue and start to explore feasibility?
> -Bryan
>
>
>
Re: Some thoughts about changes to Thrift
Posted by Esteve Fernandez <es...@sindominio.net>.
On Friday 06 March 2009 20:36:34 Mark Slee wrote:
> I do just want to call out again here that AFAIK Java tends not to be
> distributed with many flavors of Linux, whereas Python does.
I contributed a bunch of code to the Python generator, but still I think a
compiler written in Java makes sense. If we can use JIRA as an indication of
activity, the number of issues and contributors in the Java front is greater
than in the Python one.
> If a rewriting effort is undertaken, I'd just throw out a big +1 to fullly
> considering Python in addition to Java.
Do you mean writing two compilers (one in Java and the other one in Python)?
In that case, it would make more sense to use ANTLR and let it generate
parsers in different languages.
Cheers.
Re: Some thoughts about changes to Thrift
Posted by David Reiss <dr...@facebook.com>.
./bootstrap.sh
./configure
make -C compiler/cpp
The compiler requires boost, so no luck there. However, the compiler
only uses "header-only" libraries, so you shouldn't have to build
anything. Just make sure that you point the build at the boost
headers.
--David
Ted Dunning wrote:
> One big issue is that right now, it isn't clear at all how to build the
> compiler in isolation from the rest of the system. Once you commit to
> building the C language server support, then you have a bit of a barney on
> your hands because you have to build some pretty hairy things like boost.
>
> If it were more clear how to build the compiler alone, then I would find
> thrift much more portable. To be clear, this is largely because I just
> don't care about the C++ bindings.
>
> The effect is a bit similar on Mac and Linux platforms, especially if you
> don't have something like darwin ports on Mac or can't install packages at a
> systems level on Linux. You are then subject to a several hours long
> dependency download and compile exercise. The dependencies for the compiler
> alone should (presumably) be vastly simpler.
>
> 2009/3/13 David Reiss <dr...@facebook.com>
>
>> I really
>> want to know what is preventing you from building the compiler in
>> your environment of choice. Is there Linux-specific code in use?
>> Are there gaps in the documentation on the wiki? We can fix these
>> issues.
>>
>
>
>
Re: Some thoughts about changes to Thrift
Posted by Ted Dunning <te...@gmail.com>.
One big issue is that right now, it isn't clear at all how to build the
compiler in isolation from the rest of the system. Once you commit to
building the C language server support, then you have a bit of a barney on
your hands because you have to build some pretty hairy things like boost.
If it were more clear how to build the compiler alone, then I would find
thrift much more portable. To be clear, this is largely because I just
don't care about the C++ bindings.
The effect is a bit similar on Mac and Linux platforms, especially if you
don't have something like darwin ports on Mac or can't install packages at a
systems level on Linux. You are then subject to a several hours long
dependency download and compile exercise. The dependencies for the compiler
alone should (presumably) be vastly simpler.
2009/3/13 David Reiss <dr...@facebook.com>
> I really
> want to know what is preventing you from building the compiler in
> your environment of choice. Is there Linux-specific code in use?
> Are there gaps in the documentation on the wiki? We can fix these
> issues.
>
--
Ted Dunning, CTO
DeepDyve
Re: Some thoughts about changes to Thrift
Posted by David Reiss <dr...@facebook.com>.
To respond to Rush's question, no, I have never built Thrift on
Mac OS or Windows, since I don't use those systems. I don't take
your question as a challenge or a flame, but please understand that
my question was also not meant as a challenge or flame. I really
want to know what is preventing you from building the compiler in
your environment of choice. Is there Linux-specific code in use?
Are there gaps in the documentation on the wiki? We can fix these
issues.
Michael, what is stopping you from using VC++, if that is what you
prefer? Someone mentioned lex and yacc earlier. I don't think it
is a good idea to commit their generated code to the repository,
but it will be included in our releases, and I'm currently setting
up a nightly release system so you'll be able to get the latest
lex and yacc generated code.
--David
Michael Greene wrote:
> Right, that's what we are discussing here, having a version that runs in a
> non-emulated environment (i.e. not Cygwin) or that can be built using tools
> native to the environment (e.g. VC++).
> I agree that using Thrift with Cygwin is only as hard as it is in any *NIX
> environment, plus installing Cygwin.
>
> However, I need to be able to distribute the resulting executables to my
> colleagues, and the project needs to be able to easily distribute (without
> the magic of 'huh, I guess MinGW worked this time after a couple of hours of
> fiddling') native executables when it finally releases.
>
> Michael
>
> On Thu, Mar 12, 2009 at 3:54 PM, Ted Dunning <te...@gmail.com> wrote:
>
>> It isn't that it is all that hard to get a version that runs on windows.
>> If
>> you have cygwin installed, you are pretty much good to go (you may have to
>> add some packages).
>>
>> The really hard part is to give your resulting executable to anybody else
>> and expect them to run it. THAT is hard.
>>
>> On Thu, Mar 12, 2009 at 12:28 PM, Michael Greene
>> <mi...@gmail.com>wrote:
>>
>>> I don't know *why* it's a hassle, but it is. It's just a pain to set up.
>>> I'm not sure exactly why I can't build it with whatever build environment
>> I
>>> want, that's a good point. Most of what I know about MinGW and MSYS
>> comes
>>> from trying to get Thrift to work smoothly for me, and I've never been
>> able
>>> to figure out how to generate a flex lib that VC++ can link.
>>>
>>> Additionally, I follow roughly the same instructions each time (those on
>>> the
>>> wiki) but there are usually weeks between setting up different systems
>> for
>>> development internally here, and each time it's magic whether it works
>> for
>>> me in the first hour or not (or whether I give up). I realize this is
>>> vague
>>> and sounds like an ignorant response, but I've been working with Thrift
>> for
>>> awhile and this describes my experience on Windows. Based on the many
>>> messages on the lists and IRC, it's clear that others have difficulty as
>>> well.
>>>
>>> On Sat, Mar 7, 2009 at 10:14 PM, David Reiss <dr...@facebook.com>
>> wrote:
>>>> Why is it such a hassle? Why can't you build it with whatever
>>>> build environment you want?
>>>>
>>>>
Re: Some thoughts about changes to Thrift
Posted by Michael Greene <mi...@gmail.com>.
Right, that's what we are discussing here, having a version that runs in a
non-emulated environment (i.e. not Cygwin) or that can be built using tools
native to the environment (e.g. VC++).
I agree that using Thrift with Cygwin is only as hard as it is in any *NIX
environment, plus installing Cygwin.
However, I need to be able to distribute the resulting executables to my
colleagues, and the project needs to be able to easily distribute (without
the magic of 'huh, I guess MinGW worked this time after a couple of hours of
fiddling') native executables when it finally releases.
Michael
On Thu, Mar 12, 2009 at 3:54 PM, Ted Dunning <te...@gmail.com> wrote:
> It isn't that it is all that hard to get a version that runs on windows.
> If
> you have cygwin installed, you are pretty much good to go (you may have to
> add some packages).
>
> The really hard part is to give your resulting executable to anybody else
> and expect them to run it. THAT is hard.
>
> On Thu, Mar 12, 2009 at 12:28 PM, Michael Greene
> <mi...@gmail.com>wrote:
>
> > I don't know *why* it's a hassle, but it is. It's just a pain to set up.
> > I'm not sure exactly why I can't build it with whatever build environment
> I
> > want, that's a good point. Most of what I know about MinGW and MSYS
> comes
> > from trying to get Thrift to work smoothly for me, and I've never been
> able
> > to figure out how to generate a flex lib that VC++ can link.
> >
> > Additionally, I follow roughly the same instructions each time (those on
> > the
> > wiki) but there are usually weeks between setting up different systems
> for
> > development internally here, and each time it's magic whether it works
> for
> > me in the first hour or not (or whether I give up). I realize this is
> > vague
> > and sounds like an ignorant response, but I've been working with Thrift
> for
> > awhile and this describes my experience on Windows. Based on the many
> > messages on the lists and IRC, it's clear that others have difficulty as
> > well.
> >
> > On Sat, Mar 7, 2009 at 10:14 PM, David Reiss <dr...@facebook.com>
> wrote:
> >
> > > Why is it such a hassle? Why can't you build it with whatever
> > > build environment you want?
> > >
> > >
>
Re: Some thoughts about changes to Thrift
Posted by Ted Dunning <te...@gmail.com>.
It isn't that it is all that hard to get a version that runs on windows. If
you have cygwin installed, you are pretty much good to go (you may have to
add some packages).
The really hard part is to give your resulting executable to anybody else
and expect them to run it. THAT is hard.
On Thu, Mar 12, 2009 at 12:28 PM, Michael Greene
<mi...@gmail.com>wrote:
> I don't know *why* it's a hassle, but it is. It's just a pain to set up.
> I'm not sure exactly why I can't build it with whatever build environment I
> want, that's a good point. Most of what I know about MinGW and MSYS comes
> from trying to get Thrift to work smoothly for me, and I've never been able
> to figure out how to generate a flex lib that VC++ can link.
>
> Additionally, I follow roughly the same instructions each time (those on
> the
> wiki) but there are usually weeks between setting up different systems for
> development internally here, and each time it's magic whether it works for
> me in the first hour or not (or whether I give up). I realize this is
> vague
> and sounds like an ignorant response, but I've been working with Thrift for
> awhile and this describes my experience on Windows. Based on the many
> messages on the lists and IRC, it's clear that others have difficulty as
> well.
>
> On Sat, Mar 7, 2009 at 10:14 PM, David Reiss <dr...@facebook.com> wrote:
>
> > Why is it such a hassle? Why can't you build it with whatever
> > build environment you want?
> >
> > Michael Greene wrote:
> > > David Reiss <dr...@facebook.com> wrote:
> > >>> Right now, we can't do Windows because it's *impossible* for our code
> > >>> to run on that architecture due to lack of libraries. Any other
> > >>> language that runs well on a lot of platforms (Java, Ruby, Python...)
> > >>> would be fine if our only goal is cross-platform capabilities.
> > >> I don't understand. Haven't people already built the compiler on
> > >> Windows?
> > >>
> > >
> > > Yes, people have already built the compiler on Windows. It's my
> > > primary development platform. However, it is a huge hassle. Both of
> > > the building options require some form of POSIX emulation for the
> > > build environment (whether Cygwin or MinGW) and getting the
> > > dependencies right to prevent this environment being required in the
> > > resulting executable is a bit of magic. Each time I get the toolchain
> > > setup on a new Windows system I always screw something up in the
> > > process, and half the time end up leaving the Cygwin dependency out of
> > > convenience, since most of the time I only need to generate code and
> > > don't need to distribute the artifacts. It's comparable to the
> > > compiler requiring Wine to build in *NIX environments, and practically
> > > requiring Wine to run in *NIX environments.
> > >
> > > Garrett, I have also been toying with Python generation. I'd
> > > appreciate a ping if you get to any substantial milestone or even a
> > > plateau with code.
> > >
> > > Michael
> >
>
--
Ted Dunning, CTO
DeepDyve
111 West Evelyn Ave. Ste. 202
Sunnyvale, CA 94086
www.deepdyve.com
408-773-0110 ext. 738
858-414-0013 (m)
408-773-0220 (fax)
Re: Some thoughts about changes to Thrift
Posted by Michael Greene <mi...@gmail.com>.
I don't know *why* it's a hassle, but it is. It's just a pain to set up.
I'm not sure exactly why I can't build it with whatever build environment I
want, that's a good point. Most of what I know about MinGW and MSYS comes
from trying to get Thrift to work smoothly for me, and I've never been able
to figure out how to generate a flex lib that VC++ can link.
Additionally, I follow roughly the same instructions each time (those on the
wiki) but there are usually weeks between setting up different systems for
development internally here, and each time it's magic whether it works for
me in the first hour or not (or whether I give up). I realize this is vague
and sounds like an ignorant response, but I've been working with Thrift for
awhile and this describes my experience on Windows. Based on the many
messages on the lists and IRC, it's clear that others have difficulty as
well.
On Sat, Mar 7, 2009 at 10:14 PM, David Reiss <dr...@facebook.com> wrote:
> Why is it such a hassle? Why can't you build it with whatever
> build environment you want?
>
> Michael Greene wrote:
> > David Reiss <dr...@facebook.com> wrote:
> >>> Right now, we can't do Windows because it's *impossible* for our code
> >>> to run on that architecture due to lack of libraries. Any other
> >>> language that runs well on a lot of platforms (Java, Ruby, Python...)
> >>> would be fine if our only goal is cross-platform capabilities.
> >> I don't understand. Haven't people already built the compiler on
> >> Windows?
> >>
> >
> > Yes, people have already built the compiler on Windows. It's my
> > primary development platform. However, it is a huge hassle. Both of
> > the building options require some form of POSIX emulation for the
> > build environment (whether Cygwin or MinGW) and getting the
> > dependencies right to prevent this environment being required in the
> > resulting executable is a bit of magic. Each time I get the toolchain
> > setup on a new Windows system I always screw something up in the
> > process, and half the time end up leaving the Cygwin dependency out of
> > convenience, since most of the time I only need to generate code and
> > don't need to distribute the artifacts. It's comparable to the
> > compiler requiring Wine to build in *NIX environments, and practically
> > requiring Wine to run in *NIX environments.
> >
> > Garrett, I have also been toying with Python generation. I'd
> > appreciate a ping if you get to any substantial milestone or even a
> > plateau with code.
> >
> > Michael
>
Re: Some thoughts about changes to Thrift
Posted by Rush Manbert <ru...@manbert.com>.
On Mar 7, 2009, at 7:14 PM, David Reiss wrote:
> Why is it such a hassle? Why can't you build it with whatever
> build environment you want?
I'm not saying this to be challenging or to flame you, but have you
tried it? I think that if you had you'd understand what a hassle it is.
I eventually gave up, because I could run the compiler on a Mac.
>
>
> Michael Greene wrote:
>> David Reiss <dr...@facebook.com> wrote:
>>>> Right now, we can't do Windows because it's *impossible* for our
>>>> code
>>>> to run on that architecture due to lack of libraries. Any other
>>>> language that runs well on a lot of platforms (Java, Ruby,
>>>> Python...)
>>>> would be fine if our only goal is cross-platform capabilities.
>>> I don't understand. Haven't people already built the compiler on
>>> Windows?
>>>
>>
>> Yes, people have already built the compiler on Windows. It's my
>> primary development platform. However, it is a huge hassle. Both of
>> the building options require some form of POSIX emulation for the
>> build environment (whether Cygwin or MinGW) and getting the
>> dependencies right to prevent this environment being required in the
>> resulting executable is a bit of magic. Each time I get the
>> toolchain
>> setup on a new Windows system I always screw something up in the
>> process, and half the time end up leaving the Cygwin dependency out
>> of
>> convenience, since most of the time I only need to generate code and
>> don't need to distribute the artifacts. It's comparable to the
>> compiler requiring Wine to build in *NIX environments, and
>> practically
>> requiring Wine to run in *NIX environments.
>>
>> Garrett, I have also been toying with Python generation. I'd
>> appreciate a ping if you get to any substantial milestone or even a
>> plateau with code.
>>
>> Michael
Best regards,
Rush
Re: Some thoughts about changes to Thrift
Posted by David Reiss <dr...@facebook.com>.
Why is it such a hassle? Why can't you build it with whatever
build environment you want?
Michael Greene wrote:
> David Reiss <dr...@facebook.com> wrote:
>>> Right now, we can't do Windows because it's *impossible* for our code
>>> to run on that architecture due to lack of libraries. Any other
>>> language that runs well on a lot of platforms (Java, Ruby, Python...)
>>> would be fine if our only goal is cross-platform capabilities.
>> I don't understand. Haven't people already built the compiler on
>> Windows?
>>
>
> Yes, people have already built the compiler on Windows. It's my
> primary development platform. However, it is a huge hassle. Both of
> the building options require some form of POSIX emulation for the
> build environment (whether Cygwin or MinGW) and getting the
> dependencies right to prevent this environment being required in the
> resulting executable is a bit of magic. Each time I get the toolchain
> setup on a new Windows system I always screw something up in the
> process, and half the time end up leaving the Cygwin dependency out of
> convenience, since most of the time I only need to generate code and
> don't need to distribute the artifacts. It's comparable to the
> compiler requiring Wine to build in *NIX environments, and practically
> requiring Wine to run in *NIX environments.
>
> Garrett, I have also been toying with Python generation. I'd
> appreciate a ping if you get to any substantial milestone or even a
> plateau with code.
>
> Michael
Re: Some thoughts about changes to Thrift
Posted by Michael Greene <mi...@gmail.com>.
David Reiss <dr...@facebook.com> wrote:
>> Right now, we can't do Windows because it's *impossible* for our code
>> to run on that architecture due to lack of libraries. Any other
>> language that runs well on a lot of platforms (Java, Ruby, Python...)
>> would be fine if our only goal is cross-platform capabilities.
> I don't understand. Haven't people already built the compiler on
> Windows?
>
Yes, people have already built the compiler on Windows. It's my
primary development platform. However, it is a huge hassle. Both of
the building options require some form of POSIX emulation for the
build environment (whether Cygwin or MinGW) and getting the
dependencies right to prevent this environment being required in the
resulting executable is a bit of magic. Each time I get the toolchain
setup on a new Windows system I always screw something up in the
process, and half the time end up leaving the Cygwin dependency out of
convenience, since most of the time I only need to generate code and
don't need to distribute the artifacts. It's comparable to the
compiler requiring Wine to build in *NIX environments, and practically
requiring Wine to run in *NIX environments.
Garrett, I have also been toying with Python generation. I'd
appreciate a ping if you get to any substantial milestone or even a
plateau with code.
Michael
Re: Some thoughts about changes to Thrift
Posted by David Reiss <dr...@facebook.com>.
> Right now, we can't do Windows because it's *impossible* for our code
> to run on that architecture due to lack of libraries. Any other
> language that runs well on a lot of platforms (Java, Ruby, Python...)
> would be fine if our only goal is cross-platform capabilities.
I don't understand. Haven't people already built the compiler on
Windows?
Re: Some thoughts about changes to Thrift
Posted by Franis Sirkovic <fr...@gmail.com>.
I would like to remind you that thrift libraries should be unified.
I have used C#, Python and Erlang libraries and there are significant
differences between them. I suppose other libraries are different too.
For example, Erlang server will disconnect after 10 seconds (hard coded in
source) and Python server will wait forever. You can stop C# and Erlang
server (they have Stop() methods, Java server also has Stop method).
Keyboard interrupt is only way to stop Python server (I didn't find other
way).
Also, there is no way to customize C# or Java servers. There are just public
and private members. I think that there should be a way to customize classes
in all languages. For example, I have this problem because I need
notification when connection is broken. Of course, couple of standard
methods in Handler interface (new_client(), closed(), ...) can solve this
problem.
Best wishes, Franis.
On Fri, Mar 6, 2009 at 11:04 PM, Ted Dunning <te...@gmail.com> wrote:
> I think the latter.
>
> And subject to having a moderately large install, a java compiler can come
> bundled with a JRE which has no environmental dependencies other than
> platform.
>
> On Fri, Mar 6, 2009 at 11:53 AM, Bryan Duxbury <br...@rapleaf.com> wrote:
>
> > Should the goal be matching up with stuff that's already installed, or
> just
> > that when you have to install something, it's easy to do?
>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Re: Some thoughts about changes to Thrift
Posted by Ted Dunning <te...@gmail.com>.
I think the latter.
And subject to having a moderately large install, a java compiler can come
bundled with a JRE which has no environmental dependencies other than
platform.
On Fri, Mar 6, 2009 at 11:53 AM, Bryan Duxbury <br...@rapleaf.com> wrote:
> Should the goal be matching up with stuff that's already installed, or just
> that when you have to install something, it's easy to do?
--
Ted Dunning, CTO
DeepDyve
Re: Some thoughts about changes to Thrift
Posted by Bryan Duxbury <br...@rapleaf.com>.
Should the goal be matching up with stuff that's already installed,
or just that when you have to install something, it's easy to do?
Right now, we can't do Windows because it's *impossible* for our code
to run on that architecture due to lack of libraries. Any other
language that runs well on a lot of platforms (Java, Ruby, Python...)
would be fine if our only goal is cross-platform capabilities.
If we do care about whether everything is preinstalled, then I'll
throw out a vote for Ruby, just because it's my favorite option. Ruby
is often distributed with Linux and OSX, and easy to install on
WIndows and elsewhere.
-Bryan
On Mar 6, 2009, at 11:36 AM, Mark Slee wrote:
> I do just want to call out again here that AFAIK Java tends not to
> be distributed with many flavors of Linux, whereas Python does.
>
> If a rewriting effort is undertaken, I'd just throw out a big +1 to
> fullly considering Python in addition to Java.
>
> -----Original Message-----
> From: Rush Manbert [mailto:rush@manbert.com]
> Sent: Friday, March 06, 2009 10:01 AM
> To: thrift-user@incubator.apache.org
> Subject: Re: Some thoughts about changes to Thrift
>
> I don't use any command line tools that are Java-based, so I'm
> speaking from lack of experience here. :-)
>
> IMHO, a major feature of any tool implementation is that it be easy
> to integrate into user's build environments on all supported
> platforms.
> In my case, this means that I need use the tool from a Xcode
> project and also from a Visual Studio project. When we eventually
> support Linux, then I guess I'l need to use in a big Makefile-based
> set of projects. It may mean that I need to write appropriate
> makefiles and use the make-flavor of project that Xcode and VS
> support. On the Mac, in particular, this is important because I can
> make my library project that uses Thrift depend on the project that
> generates the sources using the compiler.
>
> So what I'm saying is please take this into consideration when
> picking the cross-platform language to use for rewriting the
> compiler. My personal choice would be Python, because it's nice and
> clean to use from the command line, there are template engines
> available for it, and installing Python on Windows is pretty easy.
> But if a Java-based tool has all the same characteristics, then go
> for it.
>
> - Rush
>
> On Mar 6, 2009, at 8:02 AM, Bryan Duxbury wrote:
>
>> It seems like there's some consensus on a JavaCC based compiler.
>> Should we open an JIRA issue and start to explore feasibility?
>> -Bryan
>>
>> On Mar 6, 2009, at 3:03 AM, Esteve Fernandez wrote:
>>
>>> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>>>> That's pretty much my fault.
>>>
>>> Actually the current Thrift compiler served its puporse quite
>>> decently, but the coming of new languages and features has made it
>>> (along with the
>>> generators) a bit more complex and less easy to maintain.
>>>
>>>> I personally like Java and think it'd be a decent choice, but there
>>>> do seem to be a decent number of people out there building services
>>>> who seem to vehemently hate Java. Lots of *nix systems do not
>>>> have a
>>>> JVM or JDK installed by default -- and it's pretty annoying for
>>>> users who aren't writing services in Java to have to install it to
>>>> build the compiler.
>>>
>>> I'd argue for writing a parser in Java and use a template engine
>>> (Velocity, Freemarker, etc.). Why not Python? Dunno :-) I think Java
>>> is more widely deployed, it has better tools for scanning and
>>> parsing
>>> and more developers.
>>> Etch, another incubating project with similar goals to Thrift, uses
>>> JavaCC for parsing and Velocity as template engine for its compiler
>>> and generators, even though it also emits C#
>>>
>>> Using a template engine would make development of new features and
>>> the inclusion of more languages much easier. Instead of having to
>>> hack a generator to add some new feature, we could simply supply a
>>> new template file. For example, at some point in the future I'd like
>>> to build a generator for Javascript, and if we used a template
>>> engine, it would be a matter of writing a minimal generator and a
>>> bunch of templates.
>>>
>>> Cheers.
>
RE: Some thoughts about changes to Thrift
Posted by Mark Slee <ms...@facebook.com>.
I do just want to call out again here that AFAIK Java tends not to be distributed with many flavors of Linux, whereas Python does.
If a rewriting effort is undertaken, I'd just throw out a big +1 to fullly considering Python in addition to Java.
-----Original Message-----
From: Rush Manbert [mailto:rush@manbert.com]
Sent: Friday, March 06, 2009 10:01 AM
To: thrift-user@incubator.apache.org
Subject: Re: Some thoughts about changes to Thrift
I don't use any command line tools that are Java-based, so I'm speaking from lack of experience here. :-)
IMHO, a major feature of any tool implementation is that it be easy to integrate into user's build environments on all supported platforms.
In my case, this means that I need use the tool from a Xcode project and also from a Visual Studio project. When we eventually support Linux, then I guess I'l need to use in a big Makefile-based set of projects. It may mean that I need to write appropriate makefiles and use the make-flavor of project that Xcode and VS support. On the Mac, in particular, this is important because I can make my library project that uses Thrift depend on the project that generates the sources using the compiler.
So what I'm saying is please take this into consideration when picking the cross-platform language to use for rewriting the compiler. My personal choice would be Python, because it's nice and clean to use from the command line, there are template engines available for it, and installing Python on Windows is pretty easy. But if a Java-based tool has all the same characteristics, then go for it.
- Rush
On Mar 6, 2009, at 8:02 AM, Bryan Duxbury wrote:
> It seems like there's some consensus on a JavaCC based compiler.
> Should we open an JIRA issue and start to explore feasibility?
> -Bryan
>
> On Mar 6, 2009, at 3:03 AM, Esteve Fernandez wrote:
>
>> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>>> That's pretty much my fault.
>>
>> Actually the current Thrift compiler served its puporse quite
>> decently, but the coming of new languages and features has made it
>> (along with the
>> generators) a bit more complex and less easy to maintain.
>>
>>> I personally like Java and think it'd be a decent choice, but there
>>> do seem to be a decent number of people out there building services
>>> who seem to vehemently hate Java. Lots of *nix systems do not have a
>>> JVM or JDK installed by default -- and it's pretty annoying for
>>> users who aren't writing services in Java to have to install it to
>>> build the compiler.
>>
>> I'd argue for writing a parser in Java and use a template engine
>> (Velocity, Freemarker, etc.). Why not Python? Dunno :-) I think Java
>> is more widely deployed, it has better tools for scanning and parsing
>> and more developers.
>> Etch, another incubating project with similar goals to Thrift, uses
>> JavaCC for parsing and Velocity as template engine for its compiler
>> and generators, even though it also emits C#
>>
>> Using a template engine would make development of new features and
>> the inclusion of more languages much easier. Instead of having to
>> hack a generator to add some new feature, we could simply supply a
>> new template file. For example, at some point in the future I'd like
>> to build a generator for Javascript, and if we used a template
>> engine, it would be a matter of writing a minimal generator and a
>> bunch of templates.
>>
>> Cheers.
Re: Some thoughts about changes to Thrift
Posted by Rush Manbert <ru...@manbert.com>.
I don't use any command line tools that are Java-based, so I'm
speaking from lack of experience here. :-)
IMHO, a major feature of any tool implementation is that it be easy to
integrate into user's build environments on all supported platforms.
In my case, this means that I need use the tool from a Xcode project
and also from a Visual Studio project. When we eventually support
Linux, then I guess I'l need to use in a big Makefile-based set of
projects. It may mean that I need to write appropriate makefiles and
use the make-flavor of project that Xcode and VS support. On the Mac,
in particular, this is important because I can make my library project
that uses Thrift depend on the project that generates the sources
using the compiler.
So what I'm saying is please take this into consideration when picking
the cross-platform language to use for rewriting the compiler. My
personal choice would be Python, because it's nice and clean to use
from the command line, there are template engines available for it,
and installing Python on Windows is pretty easy. But if a Java-based
tool has all the same characteristics, then go for it.
- Rush
On Mar 6, 2009, at 8:02 AM, Bryan Duxbury wrote:
> It seems like there's some consensus on a JavaCC based compiler.
> Should we open an JIRA issue and start to explore feasibility?
> -Bryan
>
> On Mar 6, 2009, at 3:03 AM, Esteve Fernandez wrote:
>
>> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>>> That's pretty much my fault.
>>
>> Actually the current Thrift compiler served its puporse quite
>> decently, but
>> the coming of new languages and features has made it (along with the
>> generators) a bit more complex and less easy to maintain.
>>
>>> I personally like Java and think it'd be a decent choice, but
>>> there do seem
>>> to be a decent number of people out there building services who
>>> seem to
>>> vehemently hate Java. Lots of *nix systems do not have a JVM or JDK
>>> installed by default -- and it's pretty annoying for users who
>>> aren't
>>> writing services in Java to have to install it to build the
>>> compiler.
>>
>> I'd argue for writing a parser in Java and use a template engine
>> (Velocity,
>> Freemarker, etc.). Why not Python? Dunno :-) I think Java is more
>> widely
>> deployed, it has better tools for scanning and parsing and more
>> developers.
>> Etch, another incubating project with similar goals to Thrift, uses
>> JavaCC
>> for parsing and Velocity as template engine for its compiler and
>> generators,
>> even though it also emits C#
>>
>> Using a template engine would make development of new features and
>> the
>> inclusion of more languages much easier. Instead of having to hack a
>> generator to add some new feature, we could simply supply a new
>> template
>> file. For example, at some point in the future I'd like to build a
>> generator
>> for Javascript, and if we used a template engine, it would be a
>> matter of
>> writing a minimal generator and a bunch of templates.
>>
>> Cheers.
Re: Some thoughts about changes to Thrift
Posted by Bryan Duxbury <br...@rapleaf.com>.
It seems like there's some consensus on a JavaCC based compiler.
Should we open an JIRA issue and start to explore feasibility?
-Bryan
On Mar 6, 2009, at 3:03 AM, Esteve Fernandez wrote:
> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>> That's pretty much my fault.
>
> Actually the current Thrift compiler served its puporse quite
> decently, but
> the coming of new languages and features has made it (along with the
> generators) a bit more complex and less easy to maintain.
>
>> I personally like Java and think it'd be a decent choice, but
>> there do seem
>> to be a decent number of people out there building services who
>> seem to
>> vehemently hate Java. Lots of *nix systems do not have a JVM or JDK
>> installed by default -- and it's pretty annoying for users who aren't
>> writing services in Java to have to install it to build the compiler.
>
> I'd argue for writing a parser in Java and use a template engine
> (Velocity,
> Freemarker, etc.). Why not Python? Dunno :-) I think Java is more
> widely
> deployed, it has better tools for scanning and parsing and more
> developers.
> Etch, another incubating project with similar goals to Thrift, uses
> JavaCC
> for parsing and Velocity as template engine for its compiler and
> generators,
> even though it also emits C#
>
> Using a template engine would make development of new features and the
> inclusion of more languages much easier. Instead of having to hack a
> generator to add some new feature, we could simply supply a new
> template
> file. For example, at some point in the future I'd like to build a
> generator
> for Javascript, and if we used a template engine, it would be a
> matter of
> writing a minimal generator and a bunch of templates.
>
> Cheers.
Re: Some thoughts about changes to Thrift
Posted by Esteve Fernandez <es...@sindominio.net>.
Hi,
Yep, not all languages are created equal. But that's what I meant when I said
that adding another language would be a matter of writing some templates and
a generator. But, the important thing is that once a minimal generator is
written, in theory, contributors would only need to write a new templates.
For example, supporting Twisted in the Python generator is actually a matter
of returning Deferreds in client calls, but it still required changes in the
generator.
Cheers.
On Friday 06 March 2009 20:28:47 Mark Slee wrote:
> I'd just point out that there are more differences between the language
> generators than it seems on the surface. We tried to have much more code
> reuse across the language generators than we ended up with, especially for
> C++/Java etc. which seem highly similar, but it turned out in practice to
> get really awkward really fast, so we erred on the side of more standalone
> language generators that are easier to understand in isolation.
>
> I still think templates with more shared code for different generators is a
> good avenue to explore, but just want to raise a caution flag that it's
> been tried, and is reasonably difficult.
>
> -----Original Message-----
> From: Esteve Fernandez [mailto:esteve@sindominio.net]
> Sent: Friday, March 06, 2009 3:04 AM
> To: thrift-user@incubator.apache.org
> Subject: Re: Some thoughts about changes to Thrift
>
> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
> > That's pretty much my fault.
>
> Actually the current Thrift compiler served its puporse quite decently, but
> the coming of new languages and features has made it (along with the
> generators) a bit more complex and less easy to maintain.
>
> > I personally like Java and think it'd be a decent choice, but there do
> > seem to be a decent number of people out there building services who
> > seem to vehemently hate Java. Lots of *nix systems do not have a JVM
> > or JDK installed by default -- and it's pretty annoying for users who
> > aren't writing services in Java to have to install it to build the
> > compiler.
>
> I'd argue for writing a parser in Java and use a template engine (Velocity,
> Freemarker, etc.). Why not Python? Dunno :-) I think Java is more widely
> deployed, it has better tools for scanning and parsing and more developers.
> Etch, another incubating project with similar goals to Thrift, uses JavaCC
> for parsing and Velocity as template engine for its compiler and
> generators, even though it also emits C#
>
> Using a template engine would make development of new features and the
> inclusion of more languages much easier. Instead of having to hack a
> generator to add some new feature, we could simply supply a new template
> file. For example, at some point in the future I'd like to build a
> generator for Javascript, and if we used a template engine, it would be a
> matter of writing a minimal generator and a bunch of templates.
>
> Cheers.
Re: Some thoughts about changes to Thrift
Posted by Bryan Duxbury <br...@rapleaf.com>.
I don't think that we're going to be able to have "one generator to
rule them all" thanks to use of templates. I do, however, think that
templates on a per-language basis will reduce the amount of generator
code needed and improve readability by a LOT.
-Bryan
On Mar 6, 2009, at 11:28 AM, Mark Slee wrote:
> I'd just point out that there are more differences between the
> language generators than it seems on the surface. We tried to have
> much more code reuse across the language generators than we ended
> up with, especially for C++/Java etc. which seem highly similar,
> but it turned out in practice to get really awkward really fast, so
> we erred on the side of more standalone language generators that
> are easier to understand in isolation.
>
> I still think templates with more shared code for different
> generators is a good avenue to explore, but just want to raise a
> caution flag that it's been tried, and is reasonably difficult.
>
> -----Original Message-----
> From: Esteve Fernandez [mailto:esteve@sindominio.net]
> Sent: Friday, March 06, 2009 3:04 AM
> To: thrift-user@incubator.apache.org
> Subject: Re: Some thoughts about changes to Thrift
>
> On Friday 06 March 2009 03:24:23 Mark Slee wrote:
>> That's pretty much my fault.
>
> Actually the current Thrift compiler served its puporse quite
> decently, but the coming of new languages and features has made it
> (along with the
> generators) a bit more complex and less easy to maintain.
>
>> I personally like Java and think it'd be a decent choice, but
>> there do
>> seem to be a decent number of people out there building services who
>> seem to vehemently hate Java. Lots of *nix systems do not have a JVM
>> or JDK installed by default -- and it's pretty annoying for users who
>> aren't writing services in Java to have to install it to build the
>> compiler.
>
> I'd argue for writing a parser in Java and use a template engine
> (Velocity, Freemarker, etc.). Why not Python? Dunno :-) I think
> Java is more widely deployed, it has better tools for scanning and
> parsing and more developers.
> Etch, another incubating project with similar goals to Thrift, uses
> JavaCC for parsing and Velocity as template engine for its compiler
> and generators, even though it also emits C#
>
> Using a template engine would make development of new features and
> the inclusion of more languages much easier. Instead of having to
> hack a generator to add some new feature, we could simply supply a
> new template file. For example, at some point in the future I'd
> like to build a generator for Javascript, and if we used a template
> engine, it would be a matter of writing a minimal generator and a
> bunch of templates.
>
> Cheers.
RE: Some thoughts about changes to Thrift
Posted by Mark Slee <ms...@facebook.com>.
I'd just point out that there are more differences between the language generators than it seems on the surface. We tried to have much more code reuse across the language generators than we ended up with, especially for C++/Java etc. which seem highly similar, but it turned out in practice to get really awkward really fast, so we erred on the side of more standalone language generators that are easier to understand in isolation.
I still think templates with more shared code for different generators is a good avenue to explore, but just want to raise a caution flag that it's been tried, and is reasonably difficult.
-----Original Message-----
From: Esteve Fernandez [mailto:esteve@sindominio.net]
Sent: Friday, March 06, 2009 3:04 AM
To: thrift-user@incubator.apache.org
Subject: Re: Some thoughts about changes to Thrift
On Friday 06 March 2009 03:24:23 Mark Slee wrote:
> That's pretty much my fault.
Actually the current Thrift compiler served its puporse quite decently, but the coming of new languages and features has made it (along with the
generators) a bit more complex and less easy to maintain.
> I personally like Java and think it'd be a decent choice, but there do
> seem to be a decent number of people out there building services who
> seem to vehemently hate Java. Lots of *nix systems do not have a JVM
> or JDK installed by default -- and it's pretty annoying for users who
> aren't writing services in Java to have to install it to build the compiler.
I'd argue for writing a parser in Java and use a template engine (Velocity, Freemarker, etc.). Why not Python? Dunno :-) I think Java is more widely deployed, it has better tools for scanning and parsing and more developers.
Etch, another incubating project with similar goals to Thrift, uses JavaCC for parsing and Velocity as template engine for its compiler and generators, even though it also emits C#
Using a template engine would make development of new features and the inclusion of more languages much easier. Instead of having to hack a generator to add some new feature, we could simply supply a new template file. For example, at some point in the future I'd like to build a generator for Javascript, and if we used a template engine, it would be a matter of writing a minimal generator and a bunch of templates.
Cheers.
Re: Some thoughts about changes to Thrift
Posted by Esteve Fernandez <es...@sindominio.net>.
On Friday 06 March 2009 03:24:23 Mark Slee wrote:
> That's pretty much my fault.
Actually the current Thrift compiler served its puporse quite decently, but
the coming of new languages and features has made it (along with the
generators) a bit more complex and less easy to maintain.
> I personally like Java and think it'd be a decent choice, but there do seem
> to be a decent number of people out there building services who seem to
> vehemently hate Java. Lots of *nix systems do not have a JVM or JDK
> installed by default -- and it's pretty annoying for users who aren't
> writing services in Java to have to install it to build the compiler.
I'd argue for writing a parser in Java and use a template engine (Velocity,
Freemarker, etc.). Why not Python? Dunno :-) I think Java is more widely
deployed, it has better tools for scanning and parsing and more developers.
Etch, another incubating project with similar goals to Thrift, uses JavaCC
for parsing and Velocity as template engine for its compiler and generators,
even though it also emits C#
Using a template engine would make development of new features and the
inclusion of more languages much easier. Instead of having to hack a
generator to add some new feature, we could simply supply a new template
file. For example, at some point in the future I'd like to build a generator
for Javascript, and if we used a template engine, it would be a matter of
writing a minimal generator and a bunch of templates.
Cheers.
RE: Some thoughts about changes to Thrift
Posted by Mark Slee <ms...@facebook.com>.
That's pretty much my fault.
The reason for choosing C++ was that installability/lack of dependencies has always been a big motivation for Thrift. Lex/yacc is more widely deployed and ready-to-go than pretty much any other scanning and parsing system out there.
I personally like Java and think it'd be a decent choice, but there do seem to be a decent number of people out there building services who seem to vehemently hate Java. Lots of *nix systems do not have a JVM or JDK installed by default -- and it's pretty annoying for users who aren't writing services in Java to have to install it to build the compiler.
We also looked at Python, but the PLY package was a separate extension to Python that was not in the standard distro, which would mean more installation woes to get Thrift up and running.
I definitely agree that the C++ compiler code is pretty ugly, and the string manipulation really nasty, but the focus with v1 was on making it easy for users of Thrift, rather than developers of Thrift.
It'll definitely be a lot of work to rewrite the compiler, but it's an effort that I would endorse provided it doesn't come at any cost to the Thrift end-user.
Cheers,
Mark
-----Original Message-----
From: Jake Luciani [mailto:jakers@gmail.com]
Sent: Thursday, March 05, 2009 5:12 PM
To: thrift-user@incubator.apache.org
Subject: Re: Some thoughts about changes to Thrift
I've always wondered why the IDL compiler was written in c++, seems like a bad choice.
I think a java compiler using JavaCC would be a bit better (though requiring java is a pain for some).
At least it would be cross platform....
-Jake
On Thu, Mar 5, 2009 at 7:15 PM, Michael Greene <mi...@gmail.com> wrote:
> Jérémie BORDIER <ah...@gmail.com> wrote:
>> I might be wrong, but i think that the current compiler isn't cross
>> platform, this could be the right time to do something to ease
>> thrift's accessibility on windows (even if they don't run the CPP
>> Library, at least they could generate Java stubs)
>
> Unfortunately, the cross-platform problems with the C++ Thrift
> compiler are mostly outside of the project's control, but I agree that
> this is a big problem at the moment. As far as I can tell (and I've
> looked far and wide) Flex and Bison versions late enough for Thrift
> simply cannot be built on Windows. I have some changes stored away in
> a working copy to build the compiler in Visual C++ that are mostly
> working, but lack a native Windows flex lib.
>
> Michael Greene
>
Re: Some thoughts about changes to Thrift
Posted by Jake Luciani <ja...@gmail.com>.
on another note is there any reason flex needs to be installed? why
not check in the resulting files for distribution?
-Jake
On Thu, Mar 5, 2009 at 8:42 PM, Mike Rettig <mi...@gmail.com> wrote:
> On Thu, Mar 5, 2009 at 7:12 PM, Jake Luciani <ja...@gmail.com> wrote:
>
>> I've always wondered why the IDL compiler was written in c++, seems
>> like a bad choice.
>>
>> I think a java compiler using JavaCC would be a bit better (though
>> requiring java is a pain for some).
>> At least it would be cross platform....
>>
>> -Jake
>>
>
> I've wondered the same myself. After all, it isn't a compiler at all. It is
> just a code generator. A cross platform language like java, ruby, perl, or
> python seems like a better choice for the "compiler". I don't see the
> benefit in c++. It seems there are a couple posts a week to the mailing
> lists about c++ build failures on various platforms.
> Anyone have an interest in contributing a java implementation? Thrift could
> benefit tremendously from a more maintainable implementation that also uses
> some text templates for each language. In Java, Velocity or Freemarker could
> take care of the code generation.
>
> I don't think the task would be monumental, but I'm not willing to take the
> task on myself. I'd be willing to contribute if someone else takes the
> initiative to start the project.
>
> Mike
>
>
>
>>
>> On Thu, Mar 5, 2009 at 7:15 PM, Michael Greene <mi...@gmail.com>
>> wrote:
>> > Jérémie BORDIER <ah...@gmail.com> wrote:
>> >> I might be wrong, but i think that the current compiler isn't cross
>> >> platform, this could be the right time to do something to ease
>> >> thrift's accessibility on windows (even if they don't run the CPP
>> >> Library, at least they could generate Java stubs)
>> >
>> > Unfortunately, the cross-platform problems with the C++ Thrift
>> > compiler are mostly outside of the project's control, but I agree that
>> > this is a big problem at the moment. As far as I can tell (and I've
>> > looked far and wide) Flex and Bison versions late enough for Thrift
>> > simply cannot be built on Windows. I have some changes stored away in
>> > a working copy to build the compiler in Visual C++ that are mostly
>> > working, but lack a native Windows flex lib.
>> >
>> > Michael Greene
>> >
>>
>
Re: Some thoughts about changes to Thrift
Posted by Mike Rettig <mi...@gmail.com>.
On Thu, Mar 5, 2009 at 7:12 PM, Jake Luciani <ja...@gmail.com> wrote:
> I've always wondered why the IDL compiler was written in c++, seems
> like a bad choice.
>
> I think a java compiler using JavaCC would be a bit better (though
> requiring java is a pain for some).
> At least it would be cross platform....
>
> -Jake
>
I've wondered the same myself. After all, it isn't a compiler at all. It is
just a code generator. A cross platform language like java, ruby, perl, or
python seems like a better choice for the "compiler". I don't see the
benefit in c++. It seems there are a couple posts a week to the mailing
lists about c++ build failures on various platforms.
Anyone have an interest in contributing a java implementation? Thrift could
benefit tremendously from a more maintainable implementation that also uses
some text templates for each language. In Java, Velocity or Freemarker could
take care of the code generation.
I don't think the task would be monumental, but I'm not willing to take the
task on myself. I'd be willing to contribute if someone else takes the
initiative to start the project.
Mike
>
> On Thu, Mar 5, 2009 at 7:15 PM, Michael Greene <mi...@gmail.com>
> wrote:
> > Jérémie BORDIER <ah...@gmail.com> wrote:
> >> I might be wrong, but i think that the current compiler isn't cross
> >> platform, this could be the right time to do something to ease
> >> thrift's accessibility on windows (even if they don't run the CPP
> >> Library, at least they could generate Java stubs)
> >
> > Unfortunately, the cross-platform problems with the C++ Thrift
> > compiler are mostly outside of the project's control, but I agree that
> > this is a big problem at the moment. As far as I can tell (and I've
> > looked far and wide) Flex and Bison versions late enough for Thrift
> > simply cannot be built on Windows. I have some changes stored away in
> > a working copy to build the compiler in Visual C++ that are mostly
> > working, but lack a native Windows flex lib.
> >
> > Michael Greene
> >
>
Re: Some thoughts about changes to Thrift
Posted by Bryan Duxbury <br...@rapleaf.com>.
Java or Ruby would be good options for new compilers. The latter
especially, since you could use ERb templates for really readable
compiler code. Unfortunately, rewriting the entire compiler might be
something of a monumental task :).
-Bryan
On Mar 5, 2009, at 5:12 PM, Jake Luciani wrote:
> I've always wondered why the IDL compiler was written in c++, seems
> like a bad choice.
>
> I think a java compiler using JavaCC would be a bit better (though
> requiring java is a pain for some).
> At least it would be cross platform....
>
> -Jake
>
> On Thu, Mar 5, 2009 at 7:15 PM, Michael Greene
> <mi...@gmail.com> wrote:
>> Jérémie BORDIER <ah...@gmail.com> wrote:
>>> I might be wrong, but i think that the current compiler isn't cross
>>> platform, this could be the right time to do something to ease
>>> thrift's accessibility on windows (even if they don't run the CPP
>>> Library, at least they could generate Java stubs)
>>
>> Unfortunately, the cross-platform problems with the C++ Thrift
>> compiler are mostly outside of the project's control, but I agree
>> that
>> this is a big problem at the moment. As far as I can tell (and I've
>> looked far and wide) Flex and Bison versions late enough for Thrift
>> simply cannot be built on Windows. I have some changes stored
>> away in
>> a working copy to build the compiler in Visual C++ that are mostly
>> working, but lack a native Windows flex lib.
>>
>> Michael Greene
>>
Re: Some thoughts about changes to Thrift
Posted by Jake Luciani <ja...@gmail.com>.
I've always wondered why the IDL compiler was written in c++, seems
like a bad choice.
I think a java compiler using JavaCC would be a bit better (though
requiring java is a pain for some).
At least it would be cross platform....
-Jake
On Thu, Mar 5, 2009 at 7:15 PM, Michael Greene <mi...@gmail.com> wrote:
> Jérémie BORDIER <ah...@gmail.com> wrote:
>> I might be wrong, but i think that the current compiler isn't cross
>> platform, this could be the right time to do something to ease
>> thrift's accessibility on windows (even if they don't run the CPP
>> Library, at least they could generate Java stubs)
>
> Unfortunately, the cross-platform problems with the C++ Thrift
> compiler are mostly outside of the project's control, but I agree that
> this is a big problem at the moment. As far as I can tell (and I've
> looked far and wide) Flex and Bison versions late enough for Thrift
> simply cannot be built on Windows. I have some changes stored away in
> a working copy to build the compiler in Visual C++ that are mostly
> working, but lack a native Windows flex lib.
>
> Michael Greene
>
Re: Some thoughts about changes to Thrift
Posted by Michael Greene <mi...@gmail.com>.
Jérémie BORDIER <ah...@gmail.com> wrote:
> I might be wrong, but i think that the current compiler isn't cross
> platform, this could be the right time to do something to ease
> thrift's accessibility on windows (even if they don't run the CPP
> Library, at least they could generate Java stubs)
Unfortunately, the cross-platform problems with the C++ Thrift
compiler are mostly outside of the project's control, but I agree that
this is a big problem at the moment. As far as I can tell (and I've
looked far and wide) Flex and Bison versions late enough for Thrift
simply cannot be built on Windows. I have some changes stored away in
a working copy to build the compiler in Visual C++ that are mostly
working, but lack a native Windows flex lib.
Michael Greene
Re: Some thoughts about changes to Thrift
Posted by Jérémie BORDIER <ah...@gmail.com>.
On Thu, Mar 5, 2009 at 10:51 PM, Esteve Fernandez <es...@sindominio.net> wrote:
> On Thursday 05 March 2009 22:34:23 Bryan Duxbury wrote:
>
> Once we had made a new release, we should definitely revamp the IDL and the
> compiler, possibly turning it into a multipass one so that it may support
> some missing features in the future (recursive structures come to mind,
> stricter validation, etc.).
>
I might be wrong, but i think that the current compiler isn't cross
platform, this could be the right time to do something to ease
thrift's accessibility on windows (even if they don't run the CPP
Library, at least they could generate Java stubs)
--
Jérémie 'ahFeel' BORDIER
Re: Some thoughts about changes to Thrift
Posted by Esteve Fernandez <es...@sindominio.net>.
On Thursday 05 March 2009 22:34:23 Bryan Duxbury wrote:
> What are everyone's thoughts on this idea?
Fine by me. However, since I think it may take some time to complete this
feature, I'd focus on making a release before April-May, so that we show
users that Thrift is making progress, we haven't done a release since the
project entered incubation.
Once we had made a new release, we should definitely revamp the IDL and the
compiler, possibly turning it into a multipass one so that it may support
some missing features in the future (recursive structures come to mind,
stricter validation, etc.).
Cheers.
Re: Some thoughts about changes to Thrift
Posted by Jérémie BORDIER <ah...@gmail.com>.
I agree with you both, Rick's suggestion sounds good.
On Thu, Mar 5, 2009 at 10:38 PM, Rick R <ri...@gmail.com> wrote:
> I'm for it, as long as there is a --non-strict compiler mode which allowed
> older messages (but printed a warning)
>
>
> On Thu, Mar 5, 2009 at 4:34 PM, Bryan Duxbury <br...@rapleaf.com> wrote:
>
>> Hey guys,
>>
>> I wanted to run an idea past everyone. There was an email to the list a few
>> days ago about Ruby enum validation of implicit values that got me thinking
>> that Thrift allows a lot of design decisions that aren't really in line with
>> best practices. In order to achieve respectable forward and backward
>> compatibility, one of Thrift's main stated goals, there are a bunch of
>> things you *should* do, but that aren't enforced. For instance, implicit
>> enum values and optional field ids in structs and on service method
>> parameters. These are things that a careful Thrift user should never really
>> do, but since they aren't enforced, there's risk for new users to do the
>> wrong thing.
>>
>> My idea is that we should change the Thrift IDL essentially to force these
>> best practices on users. The obvious benefit is that even a new user will be
>> pushed to do the right thing, avoiding potential pitfalls in writing their
>> definitions. The downside is that .thrift files will be a little more
>> verbose on average. (It's also a non-backwards-compatible change, so
>> everyone will have to update their Thrift definitions.) At least in my mind,
>> it seems like the positives would outweigh the benefits, and it would free
>> us up to do things like get rid of autogenerated negative field ids.
>>
>> What are everyone's thoughts on this idea?
>>
>> -Bryan
>>
>
>
>
> --
> We can't solve problems by using the same kind of thinking we used when we
> created them.
> - A. Einstein
>
--
Jérémie 'ahFeel' BORDIER
Re: Some thoughts about changes to Thrift
Posted by Bryan Duxbury <br...@rapleaf.com>.
I really like this idea.
On Mar 5, 2009, at 1:57 PM, Terry Jones wrote:
> Hi all
>
>>>>>> "Rick" == Rick R <ri...@gmail.com> writes:
> Rick> I'm for it, as long as there is a --non-strict compiler mode
> which
> Rick> allowed older messages (but printed a warning)
>
> I don't think I'd go that way. The main reasons are:
>
> - the Thrift code base will become unclean due to supporting the
> old and
> the new ways of doing things, with deprecation warnings.
>
> - if you let people just continue to use the more casual/lazy
> style of IDL
> spec, they will.
>
> Seeing as the IDL file already gets parsed, it wouldn't be too hard to
> write a tool that converts an old-style "lazy" IDL spec into a new-
> style
> more rigorous one (maintaining comments, whitespace, etc). That
> would give
> experienced Thrift users a way to easily upgrade. It would keep the
> code
> base clean. And it would have the desired effect on making
> newcomers write
> stricter Thrift specs.
>
> Terry
Re: Some thoughts about changes to Thrift
Posted by Terry Jones <te...@jon.es>.
Hi all
>>>>> "Rick" == Rick R <ri...@gmail.com> writes:
Rick> I'm for it, as long as there is a --non-strict compiler mode which
Rick> allowed older messages (but printed a warning)
I don't think I'd go that way. The main reasons are:
- the Thrift code base will become unclean due to supporting the old and
the new ways of doing things, with deprecation warnings.
- if you let people just continue to use the more casual/lazy style of IDL
spec, they will.
Seeing as the IDL file already gets parsed, it wouldn't be too hard to
write a tool that converts an old-style "lazy" IDL spec into a new-style
more rigorous one (maintaining comments, whitespace, etc). That would give
experienced Thrift users a way to easily upgrade. It would keep the code
base clean. And it would have the desired effect on making newcomers write
stricter Thrift specs.
Terry
Re: Some thoughts about changes to Thrift
Posted by Rick R <ri...@gmail.com>.
I'm for it, as long as there is a --non-strict compiler mode which allowed
older messages (but printed a warning)
On Thu, Mar 5, 2009 at 4:34 PM, Bryan Duxbury <br...@rapleaf.com> wrote:
> Hey guys,
>
> I wanted to run an idea past everyone. There was an email to the list a few
> days ago about Ruby enum validation of implicit values that got me thinking
> that Thrift allows a lot of design decisions that aren't really in line with
> best practices. In order to achieve respectable forward and backward
> compatibility, one of Thrift's main stated goals, there are a bunch of
> things you *should* do, but that aren't enforced. For instance, implicit
> enum values and optional field ids in structs and on service method
> parameters. These are things that a careful Thrift user should never really
> do, but since they aren't enforced, there's risk for new users to do the
> wrong thing.
>
> My idea is that we should change the Thrift IDL essentially to force these
> best practices on users. The obvious benefit is that even a new user will be
> pushed to do the right thing, avoiding potential pitfalls in writing their
> definitions. The downside is that .thrift files will be a little more
> verbose on average. (It's also a non-backwards-compatible change, so
> everyone will have to update their Thrift definitions.) At least in my mind,
> it seems like the positives would outweigh the benefits, and it would free
> us up to do things like get rid of autogenerated negative field ids.
>
> What are everyone's thoughts on this idea?
>
> -Bryan
>
--
We can't solve problems by using the same kind of thinking we used when we
created them.
- A. Einstein
Re: Some thoughts about changes to Thrift
Posted by David Reiss <dr...@facebook.com>.
FWIW, I'd be fine with dropping negative field ids and implicit enum
values.
Bryan Duxbury wrote:
> Hey guys,
>
> I wanted to run an idea past everyone. There was an email to the list
> a few days ago about Ruby enum validation of implicit values that got
> me thinking that Thrift allows a lot of design decisions that aren't
> really in line with best practices. In order to achieve respectable
> forward and backward compatibility, one of Thrift's main stated
> goals, there are a bunch of things you *should* do, but that aren't
> enforced. For instance, implicit enum values and optional field ids
> in structs and on service method parameters. These are things that a
> careful Thrift user should never really do, but since they aren't
> enforced, there's risk for new users to do the wrong thing.
>
> My idea is that we should change the Thrift IDL essentially to force
> these best practices on users. The obvious benefit is that even a new
> user will be pushed to do the right thing, avoiding potential
> pitfalls in writing their definitions. The downside is that .thrift
> files will be a little more verbose on average. (It's also a non-
> backwards-compatible change, so everyone will have to update their
> Thrift definitions.) At least in my mind, it seems like the positives
> would outweigh the benefits, and it would free us up to do things
> like get rid of autogenerated negative field ids.
>
> What are everyone's thoughts on this idea?
>
> -Bryan
Re: Some thoughts about changes to Thrift
Posted by Rush Manbert <ru...@manbert.com>.
On Mar 5, 2009, at 1:34 PM, Bryan Duxbury wrote:
> Hey guys,
>
> I wanted to run an idea past everyone. There was an email to the
> list a few days ago about Ruby enum validation of implicit values
> that got me thinking that Thrift allows a lot of design decisions
> that aren't really in line with best practices. In order to achieve
> respectable forward and backward compatibility, one of Thrift's main
> stated goals, there are a bunch of things you *should* do, but that
> aren't enforced. For instance, implicit enum values and optional
> field ids in structs and on service method parameters. These are
> things that a careful Thrift user should never really do, but since
> they aren't enforced, there's risk for new users to do the wrong
> thing.
>
> My idea is that we should change the Thrift IDL essentially to force
> these best practices on users. The obvious benefit is that even a
> new user will be pushed to do the right thing, avoiding potential
> pitfalls in writing their definitions. The downside is that .thrift
> files will be a little more verbose on average. (It's also a non-
> backwards-compatible change, so everyone will have to update their
> Thrift definitions.) At least in my mind, it seems like the
> positives would outweigh the benefits, and it would free us up to do
> things like get rid of autogenerated negative field ids.
>
> What are everyone's thoughts on this idea?
>
> -Bryan
I think this sounds good. It's good to help people not shoot
themselves in the foot.
- Rush