You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by Ralph Johnson <jo...@cs.uiuc.edu> on 2016/02/20 13:19:13 UTC

Advice on compiler bug workaround

I'm working on a large Groovy project that is several years old.   We've
upgraded the Groovy compiler many times.  I upgraded to the last version
three or four months ago.

The system has three major packages.  There is a core library and two
applications that use it; a web app and a Swing app.  The library is quite
a bit larger than the apps.

In the past, I would sometime run into compiler errors.   I use Eclipse,
and Eclipse is controlling the build.   When this happened, I'd tell it to
clean all projects.  Each of the three packages I mentioned is a separate
Eclipse project.  Sometimes cleaning projects worked, sometimes it took a
bit of fiddling to figure out how to get around the error.

I've been happy with the latest version of Groovy, and hadn't had any
compiler problems with it til yesterday.  After 4-6 hours of fiddling, I am
still stuck.  So, I am trying to get ideas about what to do next.

The main problem is with the web app.  When I clean and rebuild, it ends up
in one of several states.  One state is where two of the files (out of
about 80) have errors, but the particular files change and the errors are
weird.   One file has dozens of errors, all of the form "Unexpected MOP
methods in the class node for com.metaficient.xxx.YYY(super$2$toString)"
where the xxx.YYY is always a different class in the web app (not in the
much bigger library), the number 2 is usually a different small integer,
and "toString" is often "toString" but often a different method name, like
"getStaticMetaClass", "setReadOnly", or "addItem".  The other file has just
one error, "internal compiler error".

Another state is where it tells me it can't build the library and so can't
build the apps.  But then it doesn't tell me anything about why it can't
build the library.

The system was working in the morning, and then it broke.  I know which
class I was editing, so I can go back to an earlier version if necessary.
However, the code was boring.   I haven't written any new metaprogramming
code for months, so I am pretty sure the "Unexpected MOP methods" error are
not my fault.  The error messages never happen in the classes I changed
yesterday, but always in classes I haven't changed for months.

Any suggestions about what to try next?

-Ralph Johnson

Re: Advice on compiler bug workaround

Posted by Ralph Johnson <jo...@cs.uiuc.edu>.
I understand now what "Unexpected MOP methods" means.  However, I had not
changed the class hierarchy for a few days.  Moreover, I would "clean" the
system, which means that all the classes were recompiled.   So, the problem
is not because I changed the class hierarchy.  Moreover, when I recompile
everything, a different pair of classes would report the errors, so you are
almost certainly right that the problem is caused by the Eclipse build
system and I should use another build tool.   I am trying to use Gradle but
I have never used it before and this is a fairly complex project so it
might take awhile to get it all working.

The change I made on the last day was to copy down a method from a
superclass.   The superclass was in Java.  It was from Vaadin.   The
subclass overrides a lot of methods from the superclass, and in fact should
probably be rewritten to not be a subclass.  I realized that I needed to
change one of the superclass methods, but didn't want to do that.  So, I
copied it down to the subclass.

The method is setItemDataSource, and there was already a setItemDataSource
in the subclass that had "super.setItemDataSource".  So, when I copied the
method down, I gave it a new name "privateSetItemDataSource".  Then I
realized that it was accessing the private variable, "itemDataSource",
which was no longer accessible, and so tried to access it by calling the
version of setItemDataSource in the superclass.  I had the line
  super.itemDataSource = newItemDataSource
and did not get a compile error, but it seemed to cause the error with
unexpected MOP methods.  When I replaced it with
  super.setItemDataSource(newItemDataSource)
then the compile errors went away.

It is compiling better now, but I am still going to get the Gradle build
working

Thanks!

-Ralph Johnson

On Sat, Feb 20, 2016 at 9:35 AM, Jochen Theodorou <bl...@gmx.org> wrote:

> On 20.02.2016 13:19, Ralph Johnson wrote:
> [...]
>
>> One file has dozens of errors, all of the form "Unexpected
>> MOP methods in the class node for
>> com.metaficient.xxx.YYY(super$2$toString)" where the xxx.YYY is always a
>> different class in the web app (not in the much bigger library), the
>> number 2 is usually a different small integer, and "toString" is often
>> "toString" but often a different method name, like "getStaticMetaClass",
>> "setReadOnly", or "addItem".  The other file has just one error,
>> "internal compiler error".
>>
>
> first an explanation what super$2$toString means and what it was/is used
> for. Groovy needs to invoke methods, but prior to invokedynamic there is no
> means of invoking a method from outside, you have overridden and where you
> want to invoke the overriden version.
>
> Example, you make a class extending Object, in which you overwrite
> toString() and in there you call super.toString() to get the object
> variant. Would we use reflection to simulate the toSring call we would end
> up in a stack overflow error, since we would not be able to call
> Object#toString, and would always end up calling our new toString. This is
> why we have those super$X$M methods, where X should be the "distance" to
> Object, and M stands for the method. These methods do a special invocation
> just like the JVM would do using the invokespecial bytecode (such an
> invocation is normally only legal in the same class hierarchy). In our
> example you would get something like super$1$toString() then.
>
> This logic has one important drawback, if you add or remove an ancestor,
> you get into trouble.So in Java if you have A extends B, you can easily
> make a change in which B will then extend C, without having to recompile A
> in many cases. In Groovy this will not work, since it now can happen, that
> the call to such a super method will call the wrong method in the hierarchy.
>
> Now imagine A extends B extends C extends D, and you remove C. removing B
> would require recompilation in Java as well, but in Groovy removing C will
> require you recompiling A as well.
>
> Now.. how do we get to "Unexpected MOP methods"? Most likely it means the
> hierarchy changed by the removal or addition of a class and you did not yet
> recompile all needed classes. Remember, in a case of A extends B extends C
> extends D, removing D will require the compilation of C in Java and Groovy,
> but in Groovy also of A and B. And if you don't recompile them together,
> you need first to recompile C, then B and then A. Since you do not compile
> using a build tool it seems, it is possible, that Eclipse tries to compile
> in the wrong order here... especially if the Java rules are applied only.
>
>  The other file has just one error, "internal compiler error".
>>
>
> that sounds like a bug in the compiler
>
> Another state is where it tells me it can't build the library and so
>> can't build the apps.  But then it doesn't tell me anything about why it
>> can't build the library.
>>
>
> Without error message I cannot do much... but I do advice using a build
> tool outside of the IDE. You would get an error message there.
>
> The system was working in the morning, and then it broke.  I know which
>> class I was editing, so I can go back to an earlier version if
>> necessary. However, the code was boring.
>>
>
> Well, what kind of change did you do?
>
> bye Jochen
>

Re: Advice on compiler bug workaround

Posted by Jochen Theodorou <bl...@gmx.org>.
On 20.02.2016 13:19, Ralph Johnson wrote:
[...]
> One file has dozens of errors, all of the form "Unexpected
> MOP methods in the class node for
> com.metaficient.xxx.YYY(super$2$toString)" where the xxx.YYY is always a
> different class in the web app (not in the much bigger library), the
> number 2 is usually a different small integer, and "toString" is often
> "toString" but often a different method name, like "getStaticMetaClass",
> "setReadOnly", or "addItem".  The other file has just one error,
> "internal compiler error".

first an explanation what super$2$toString means and what it was/is used 
for. Groovy needs to invoke methods, but prior to invokedynamic there is 
no means of invoking a method from outside, you have overridden and 
where you want to invoke the overriden version.

Example, you make a class extending Object, in which you overwrite 
toString() and in there you call super.toString() to get the object 
variant. Would we use reflection to simulate the toSring call we would 
end up in a stack overflow error, since we would not be able to call 
Object#toString, and would always end up calling our new toString. This 
is why we have those super$X$M methods, where X should be the "distance" 
to Object, and M stands for the method. These methods do a special 
invocation just like the JVM would do using the invokespecial bytecode 
(such an invocation is normally only legal in the same class hierarchy). 
In our example you would get something like super$1$toString() then.

This logic has one important drawback, if you add or remove an ancestor, 
you get into trouble.So in Java if you have A extends B, you can easily 
make a change in which B will then extend C, without having to recompile 
A in many cases. In Groovy this will not work, since it now can happen, 
that the call to such a super method will call the wrong method in the 
hierarchy.

Now imagine A extends B extends C extends D, and you remove C. removing 
B would require recompilation in Java as well, but in Groovy removing C 
will require you recompiling A as well.

Now.. how do we get to "Unexpected MOP methods"? Most likely it means 
the hierarchy changed by the removal or addition of a class and you did 
not yet recompile all needed classes. Remember, in a case of A extends B 
extends C extends D, removing D will require the compilation of C in 
Java and Groovy, but in Groovy also of A and B. And if you don't 
recompile them together, you need first to recompile C, then B and then 
A. Since you do not compile using a build tool it seems, it is possible, 
that Eclipse tries to compile in the wrong order here... especially if 
the Java rules are applied only.

>  The other file has just one error, "internal compiler error".

that sounds like a bug in the compiler

> Another state is where it tells me it can't build the library and so
> can't build the apps.  But then it doesn't tell me anything about why it
> can't build the library.

Without error message I cannot do much... but I do advice using a build 
tool outside of the IDE. You would get an error message there.

> The system was working in the morning, and then it broke.  I know which
> class I was editing, so I can go back to an earlier version if
> necessary. However, the code was boring.

Well, what kind of change did you do?

bye Jochen