a new joint compiler

[Jochen Theodorou <bl...@gmx.org>]

Hi all,

During transit I am these days working on a new joint compiler for 
Groovy, one that handles Groovy transforms correctly and does not 
require stubs (we can still create them).

The current current version works for several cases and is still 
incomplete, but there are some decisions to make in which I would like 
to ask the people here about their opinion.

The main point is actually about keeping the code in Groovy or not and 
if to add the code to the Groovy codebase or not.

A Groovy based joint compiler will not be easy to integrate in our 
build, if it is supposed to run with current Groovy. The joint compiler 
on the other hand does not require any changes to the current compiler, 
even though it could benefit from them. So in theory it would be 
possible to use any Groovy 2.x with this joint compiler. I think that 
could be interesting as well. Though I have done nothing about build 
tools so far. At the very least the best way of integrating them will 
require some thought - but I am not sure that just replacing what we 
currently have in Groovy is the best way here. And I do like the idea of 
using Groovy to build Groovy ;)

Changing the codebase to Java will surely at least double the lines of 
code and some logic changes since I do require double dispatch in many 
places. But I am still in a proof of concept phase, so there will 
naturally be many lines of code more in the future as well.

And of course, if the integration of such a compiler is not wished for, 
it would naturally become its own project.

So what do other people think about that?

bye Jochen

Re: a new joint compiler

[Alessio Stalla <al...@gmail.com>]

Then it is a bootstrapping issue. You can solve it either using an older
binary version of Groovy to build the new version of Groovy, or writing
enough of a Groovy interpreter in Java that you can build your compiler
with it.

On 23 February 2016 at 12:47, Jochen Theodorou <bl...@gmx.org> wrote:

>
>
> On 23.02.2016 11:03, Alessio Stalla wrote:
>
>> Why is it hard to add to the build? Is it a bootstrapping issue?
>>
>
> right now we have 2 steps, build the compiler, then use the new compiler
> to build tests and runtime as well as modules. Since I need part of the
> runtime for the compiler in my case, it cannot work like this, unless
> enough of the runtime is Java to run the parts of the new compiler I am
> using. This should be the case, but can be trouble in the future.
>
> bye Jochen
>

Re: a new joint compiler

[Jochen Theodorou <bl...@gmx.org>]

On 23.02.2016 11:03, Alessio Stalla wrote:
> Why is it hard to add to the build? Is it a bootstrapping issue?

right now we have 2 steps, build the compiler, then use the new compiler 
to build tests and runtime as well as modules. Since I need part of the 
runtime for the compiler in my case, it cannot work like this, unless 
enough of the runtime is Java to run the parts of the new compiler I am 
using. This should be the case, but can be trouble in the future.

bye Jochen

Re: a new joint compiler

[Alessio Stalla <al...@gmail.com>]

Why is it hard to add to the build? Is it a bootstrapping issue?

On 23 February 2016 at 09:26, Cédric Champeau <ce...@gmail.com>
wrote:

> Hi Jochen,
>
> I need more context too. What changes are you talking about? It seems very
> abstract so far. I would be in favor of a joint compiler without stubs in
> Groovy core itself. I think both Gradle and Jetbrains would be interested
> in such a compiler too. And not talking about an incremental compiler.
> What, technically, are the necessary changes?
>
> 2016-02-23 9:18 GMT+01:00 Thibault Kruse <ti...@googlemail.com>:
>
>> I needed some background information, I guess that is given in :
>>
>> http://blackdragsview.blogspot.de/2014/11/a-joint-compiler-for-groovy-and-java.html
>> http://blackdragsview.blogspot.de/2007/07/joint-compilation-in-groovy.html
>>
>> http://wiki.jvmlangsummit.com/images/8/8a/Clement_MixedLanguageProjectCompilationInEclipse.pdf
>>
>> I believe if the joint compiler has good chances of replacing the
>> current compiler it should live in Groovy core. If it is doomed to
>> forever remain an unloved twin, it should be a project of it's own.
>>
>> In the mean time it can either be a separate project with similar
>> package names (for easy migrationinto groovy later) , or an
>> experimental branch of groovy.
>>
>> On Tue, Feb 23, 2016 at 8:37 AM, Jochen Theodorou <bl...@gmx.org>
>> wrote:
>> > Hi all,
>> >
>> > During transit I am these days working on a new joint compiler for
>> Groovy,
>> > one that handles Groovy transforms correctly and does not require stubs
>> (we
>> > can still create them).
>> >
>> > The current current version works for several cases and is still
>> incomplete,
>> > but there are some decisions to make in which I would like to ask the
>> people
>> > here about their opinion.
>> >
>> > The main point is actually about keeping the code in Groovy or not and
>> if to
>> > add the code to the Groovy codebase or not.
>> >
>> > A Groovy based joint compiler will not be easy to integrate in our
>> build, if
>> > it is supposed to run with current Groovy. The joint compiler on the
>> other
>> > hand does not require any changes to the current compiler, even though
>> it
>> > could benefit from them. So in theory it would be possible to use any
>> Groovy
>> > 2.x with this joint compiler. I think that could be interesting as well.
>> > Though I have done nothing about build tools so far. At the very least
>> the
>> > best way of integrating them will require some thought - but I am not
>> sure
>> > that just replacing what we currently have in Groovy is the best way
>> here.
>> > And I do like the idea of using Groovy to build Groovy ;)
>> >
>> > Changing the codebase to Java will surely at least double the lines of
>> code
>> > and some logic changes since I do require double dispatch in many
>> places.
>> > But I am still in a proof of concept phase, so there will naturally be
>> many
>> > lines of code more in the future as well.
>> >
>> > And of course, if the integration of such a compiler is not wished for,
>> it
>> > would naturally become its own project.
>> >
>> > So what do other people think about that?
>> >
>> > bye Jochen
>>
>
>

Re: a new joint compiler

[Jochen Theodorou <bl...@gmx.org>]

If there are no further comments in this thread I will do this as 
separate project for now. It can still be included later on

bye Jochen

Re: a new joint compiler

[Jochen Theodorou <bl...@gmx.org>]

On 23.02.2016 09:26, Cédric Champeau wrote:
> Hi Jochen,
>
> I need more context too. What changes are you talking about? It seems
> very abstract so far. I would be in favor of a joint compiler without
> stubs in Groovy core itself. I think both Gradle and Jetbrains would be
> interested in such a compiler too. And not talking about an incremental
> compiler. What, technically, are the necessary changes?

To be frank it just occurred to me that this is actually not a joint 
compiler in the sense, that it compiles Groovy and Java at the same 
time. Instead I am using a Java parser (Github project javaparser) to 
parse Java, produce a Groovy AST from it (including some semantic 
information like resolved classes) and then compile our Groovy classes 
against this. An actual Java compiler could be in a completely separate 
process.

Further plans include producing a javax.model shadow/facade AST, that 
allows annotation processors to run on Groovy code (of course without 
changing the class), a javadoc style tool with doclet support which can 
handle Groovy and Java at the same time and maybe actual joint 
compilation with other compilers through javax.model, some kind of IR or 
something entirely different I have not thought of yet.

In other words, this little project works quite different from the old 
approach of producing stubs from Groovy, run java on the stubs, run 
Groovy on the produced classes. Stubs would become an optional extra. We 
can probably fiddle most of the current logic into the new one in terms 
of ant tasks, and maybe maven2. For Gradle we would have to take a good 
look at the groovy plugin before I can tell what needs to change. And 
Jetbrains is an entirely different topic...

bye Jochen

Re: a new joint compiler

[Cédric Champeau <ce...@gmail.com>]

Hi Jochen,

I need more context too. What changes are you talking about? It seems very
abstract so far. I would be in favor of a joint compiler without stubs in
Groovy core itself. I think both Gradle and Jetbrains would be interested
in such a compiler too. And not talking about an incremental compiler.
What, technically, are the necessary changes?

2016-02-23 9:18 GMT+01:00 Thibault Kruse <ti...@googlemail.com>:

> I needed some background information, I guess that is given in :
>
> http://blackdragsview.blogspot.de/2014/11/a-joint-compiler-for-groovy-and-java.html
> http://blackdragsview.blogspot.de/2007/07/joint-compilation-in-groovy.html
>
> http://wiki.jvmlangsummit.com/images/8/8a/Clement_MixedLanguageProjectCompilationInEclipse.pdf
>
> I believe if the joint compiler has good chances of replacing the
> current compiler it should live in Groovy core. If it is doomed to
> forever remain an unloved twin, it should be a project of it's own.
>
> In the mean time it can either be a separate project with similar
> package names (for easy migrationinto groovy later) , or an
> experimental branch of groovy.
>
> On Tue, Feb 23, 2016 at 8:37 AM, Jochen Theodorou <bl...@gmx.org>
> wrote:
> > Hi all,
> >
> > During transit I am these days working on a new joint compiler for
> Groovy,
> > one that handles Groovy transforms correctly and does not require stubs
> (we
> > can still create them).
> >
> > The current current version works for several cases and is still
> incomplete,
> > but there are some decisions to make in which I would like to ask the
> people
> > here about their opinion.
> >
> > The main point is actually about keeping the code in Groovy or not and
> if to
> > add the code to the Groovy codebase or not.
> >
> > A Groovy based joint compiler will not be easy to integrate in our
> build, if
> > it is supposed to run with current Groovy. The joint compiler on the
> other
> > hand does not require any changes to the current compiler, even though it
> > could benefit from them. So in theory it would be possible to use any
> Groovy
> > 2.x with this joint compiler. I think that could be interesting as well.
> > Though I have done nothing about build tools so far. At the very least
> the
> > best way of integrating them will require some thought - but I am not
> sure
> > that just replacing what we currently have in Groovy is the best way
> here.
> > And I do like the idea of using Groovy to build Groovy ;)
> >
> > Changing the codebase to Java will surely at least double the lines of
> code
> > and some logic changes since I do require double dispatch in many places.
> > But I am still in a proof of concept phase, so there will naturally be
> many
> > lines of code more in the future as well.
> >
> > And of course, if the integration of such a compiler is not wished for,
> it
> > would naturally become its own project.
> >
> > So what do other people think about that?
> >
> > bye Jochen
>

Re: a new joint compiler

[Thibault Kruse <ti...@googlemail.com>]

I needed some background information, I guess that is given in :
http://blackdragsview.blogspot.de/2014/11/a-joint-compiler-for-groovy-and-java.html
http://blackdragsview.blogspot.de/2007/07/joint-compilation-in-groovy.html
http://wiki.jvmlangsummit.com/images/8/8a/Clement_MixedLanguageProjectCompilationInEclipse.pdf

I believe if the joint compiler has good chances of replacing the
current compiler it should live in Groovy core. If it is doomed to
forever remain an unloved twin, it should be a project of it's own.

In the mean time it can either be a separate project with similar
package names (for easy migrationinto groovy later) , or an
experimental branch of groovy.

On Tue, Feb 23, 2016 at 8:37 AM, Jochen Theodorou <bl...@gmx.org> wrote:
> Hi all,
>
> During transit I am these days working on a new joint compiler for Groovy,
> one that handles Groovy transforms correctly and does not require stubs (we
> can still create them).
>
> The current current version works for several cases and is still incomplete,
> but there are some decisions to make in which I would like to ask the people
> here about their opinion.
>
> The main point is actually about keeping the code in Groovy or not and if to
> add the code to the Groovy codebase or not.
>
> A Groovy based joint compiler will not be easy to integrate in our build, if
> it is supposed to run with current Groovy. The joint compiler on the other
> hand does not require any changes to the current compiler, even though it
> could benefit from them. So in theory it would be possible to use any Groovy
> 2.x with this joint compiler. I think that could be interesting as well.
> Though I have done nothing about build tools so far. At the very least the
> best way of integrating them will require some thought - but I am not sure
> that just replacing what we currently have in Groovy is the best way here.
> And I do like the idea of using Groovy to build Groovy ;)
>
> Changing the codebase to Java will surely at least double the lines of code
> and some logic changes since I do require double dispatch in many places.
> But I am still in a proof of concept phase, so there will naturally be many
> lines of code more in the future as well.
>
> And of course, if the integration of such a compiler is not wished for, it
> would naturally become its own project.
>
> So what do other people think about that?
>
> bye Jochen