You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Jochen Theodorou <bl...@gmx.org> on 2015/10/08 13:08:27 UTC
challenges through Java modules (aka jigsaw)
Hi all,
With Jigsaw here will be some changes affecting Groovy in some major
ways, but frankly it is still unclear to me as of how much. This is
partially to Jigsaw still being so young. This mail is mostly to write
down some thoughts for sharing.
Jigsaw means that we will not have access to all classes we have now.
Many factory methods of the JDK for example will provide such classes.
Jigsaw means we can still get a class object from it, but we cannot get
the methods or do invocations directly with the implementation class.
Things need to get through the public base class.
This is for example a problem with flow typing in static compilation:
class Base { def foo(){}; static baseImpl(){return new Hidden()}}
class Hidden extends Base {}
Hidden h = Base.baseImpl() //works
h.foo() // works not
Base b = h
b.foo() // works (in Java at least)
h.foo() fails in Java because Hidden is used as base class and protected
by the module system. Doing the same invocation using the public
available class will work though. This of course will here still invoke
Hidden#foo thanks to it being a virtual method call.
For static compilation the problem is, that the trick with Base b might
not work. b will still have the flow type Hidden. We need to ensure we
do the invocation not with the flow type, if the method is also
available in the normal static type.
But now that I think of it, the compiler might already have a problem,
since the compiler can probably not inspect Hidden by Reflection at all.
This means we need to check the code generating ClassNodes how it reacts
to exceptions being thrown due to the class object being available, but
not the members.
This also means, that if we look for a class sing reflection like
Unsafe, and assume, just because the class is there, that we can use it,
we have a code part to change. I am thinking of Unsafe hereof course,
but also about that special interface we use for callsite caching, that
bypasses verification. There is probably more.
And of course the Groovy runtime has similar problems. We have already
some checks for classes not being available through strict
SecurityManagers, but new exceptions (JDK9 specific ones) can be a
problem.. and most probably several places are involved
Next problem is with @Grab. It looks like there will be no longer a
URLClassLoader from the JDK available. Thus we can no longer add a jar
at runtime. RootLoader (used when executing Groovy from the console) can
still be used of course, but does not cover the system loader case,
which databses drivers may need. It is yet unclear to me how the module
system works at runtime with database drivers added at runtime as well.
Then I see a conceptual problem for Groovy as well. If a module is
supposed to be written in Groovy and has a private part, then the Groovy
runtime cannot really access those classes and for example cannot call
private methods in those classes. To give the runtime access I think
Groovy itself would need a module descriptor. But unless I misunderstand
the proposal, this means a random script can access those private
classes from that module written in Groovy as well. Why? Because, if it
does a method call through the runtime, and if the runtime has access,
then the script has access.
There is a concept called Layers, which I have not yet found enough
information about, which may help here... not sure yet.
bye blackdrag
--
Jochen "blackdrag" Theodorou
blog: http://blackdragsview.blogspot.com/
Re: challenges through Java modules (aka jigsaw)
Posted by Guillaume Laforge <gl...@gmail.com>.
Furthermore, there are other languages like Scala which broke people's code
at every release, and they are still being used and haven't seen that much
fragmentation.
On Fri, Nov 27, 2015 at 7:17 PM, Russel Winder <ru...@winder.org.uk> wrote:
> On Thu, 2015-11-26 at 21:05 +0100, Guillaume Laforge wrote:
> […]
> > compatibility, but the differences between Python 2 and 3 were much
> > bigger
> > that what we're speaking about here.
>
> It is time to put this Python 2 to Python 3 bogeyman into the dustbin
> of history. With Python 3.3 and later conversion of Python 2.7 codes to
> Python 3.x (x ≥ 3) codes is relatively straightforward unless your code
> is fundamentally based on a string of length 1 being a 8-bit byte
> (which is why Mercurial has a problem). No other Python 2 codes have a
> problem other than intransigence of the programmers.
>
> --
> Russel.
>
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip:
> sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC member
Product Ninja & Advocate at Restlet <http://restlet.com>
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Thu, 2015-11-26 at 21:05 +0100, Guillaume Laforge wrote:
[…]
> compatibility, but the differences between Python 2 and 3 were much
> bigger
> that what we're speaking about here.
It is time to put this Python 2 to Python 3 bogeyman into the dustbin
of history. With Python 3.3 and later conversion of Python 2.7 codes to
Python 3.x (x ≥ 3) codes is relatively straightforward unless your code
is fundamentally based on a string of length 1 being a 8-bit byte
(which is why Mercurial has a problem). No other Python 2 codes have a
problem other than intransigence of the programmers.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Sat, 2015-11-28 at 10:32 +0100, Jochen Theodorou wrote:
> […]
>
> I think Russel is more targeting the way things work in JDK8. Which
> means going away from collections and embrace streams instead for
> example.
Jochen has "hit the nail on the head". I was perhaps a little vague in
my original comments. Let me expand.
GPars 1.x is a JDK7+ system that relies on a vendor fork of extra166y,
this provides the ParallelArray type on which GPars relies for
implementing section of the overall functionality. As we now know this
route to parallelism was abandoned by JDK in favour of Streams. This is
a Good Thing™. It means though that a significant part of GPars is
built on stuff that has no future and indeed GPars in not making use of
future looking infrastructure.
Of course people can use Streams in Groovy without GPars as long as
they are on JDK8+.
I have created a new branch of GPars, which should be GPars 2.0 when
ready, which removes all the ParallelArray stuff and the vendor fork of
extra166y. Whilst a lot of GPars still works, large parts of the
overall API now do not work and/or have been removed. One strategy is
to go this route and have people use Streams much more explicitly in
Groovy and have GPars only provide the dataflow, CSP and actors stuff.
There are lots of positive reasons for going this route, not least that
it makes GPars smaller and easier to maintain whilst providing the
really important parallelism features in a post JDK7 world. However
many have indicated they prefer the old GPars API compared to the Java
Streams API and would like to see the GPars 1.3 API reimplemented over
Streams rather than using ParallelArray. This is the current state of
GPars 2.0, ParallelArray is gone, but the 1.3 API is not reimplemented
in Streams.
What we need is for people to pick up a few bits of the old GPars 1.3
API that relied on ParallelArray and propose a reimplementation in
Streams. I may be able to get stuck in to this after next week, but all
help is most welcome.
Also, if anyone finds any problems in GPars 1.3 please send in pull
requests to get things fixed – we will be maintaining 1.3 for a while
as GPars 2.0 is not ready for release and there are still recalcitrants
using JDK1.x where x < 8.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 28.11.2015 07:36, Balachandran Sivakumar wrote:
> Hi Russel,
>
> On Fri, Nov 27, 2015 at 06:22:33PM +0000, Russel Winder wrote:
>>
>> Sadly, currently the JDK8 version of GPars is in a mess, so the JDK7
>> version is all that works. I suspect 4 people in a days work could fix
>> almost all of it, but will such a hackathon happen at Groovy and Grails
>> eXchange 2015?
>>
>
> I can pitch in through github PRs. But from what I see, I can build GPars with both
> OpenJDK8 and Sun JDK8. I do see a few warnings and I'll try to resolve those over the
> next few days. But what exactly is wrong with GPars + JDK8 ? Thanks
I think Russel is more targeting the way things work in JDK8. Which
means going away from collections and embrace streams instead for example.
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Balachandran Sivakumar <ba...@balachandran.org>.
Hi Russel,
On Fri, Nov 27, 2015 at 06:22:33PM +0000, Russel Winder wrote:
>
> Sadly, currently the JDK8 version of GPars is in a mess, so the JDK7
> version is all that works. I suspect 4 people in a days work could fix
> almost all of it, but will such a hackathon happen at Groovy and Grails
> eXchange 2015?
>
I can pitch in through github PRs. But from what I see, I can build GPars with both
OpenJDK8 and Sun JDK8. I do see a few warnings and I'll try to resolve those over the
next few days. But what exactly is wrong with GPars + JDK8 ? Thanks
--
Thank you,
Balachandran Sivakumar.
> --
> Russel.
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 28.11.2015 09:12, Thibault Kruse wrote:
> Given the funding of groovy changed last year, this situation will show
> what kind of support exists. I know my PRs are growing long beards.
> Maybe work on this could be modularized in smaller milestones, and be
> pursues over kickstarter or GSoC.
We will absolutely need small milestones for this... just so people know
what the most important target right now is.
As for support... it can be as good as any other open source project,
that is not having a single paid developer.
As for kickstarter. I am curious to see who would be even able to do
work with that money. On my side at least it does not look like it is
very possible. Others might be in a more flexible position.
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
We have builds setup for the latest JDK 9 and JDK 8 forests. Those builds
help us capture bugs early. Unfortunately, we cannot build Groovy with
Jigsaw anymore (requires changes to Gradle, that requires changes to
Groovy, how fun is that?)
2015-11-28 10:36 GMT+01:00 Jochen Theodorou <bl...@gmx.org>:
> On 28.11.2015 10:01, Thibault Kruse wrote:
>
>> Also, before starting refactorings it might be usefule to put up
>> oracle JDK builds on TeamCity.
>> I have no idea why that's not already being done, but it does not seem
>> wise to me.
>>
>
> Not sure what you mean... I thought we do that
>
> bye blackdrag
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 28.11.2015 10:01, Thibault Kruse wrote:
> Also, before starting refactorings it might be usefule to put up
> oracle JDK builds on TeamCity.
> I have no idea why that's not already being done, but it does not seem
> wise to me.
Not sure what you mean... I thought we do that
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Thibault Kruse <ti...@googlemail.com>.
Also, before starting refactorings it might be usefule to put up
oracle JDK builds on TeamCity.
I have no idea why that's not already being done, but it does not seem
wise to me.
On Sat, Nov 28, 2015 at 9:12 AM, Thibault Kruse
<ti...@googlemail.com> wrote:
> Given the funding of groovy changed last year, this situation will show what
> kind of support exists. I know my PRs are growing long beards. Maybe work on
> this could be modularized in smaller milestones, and be pursues over
> kickstarter or GSoC.
>
> Also I remember another discussion where I put forward the idea of reducing
> the core codebase. Meaning to split out things like groovysh.
>
> It's not ideal maybe, but this might be a 'pick the lesser of two evils'
> situation.
>
>
> On Thursday, November 26, 2015, Guillaume Laforge <gl...@gmail.com>
> wrote:
>>
>> I'm also thinking it's the right moment to "fix" things we've done wrong,
>> have a clean separation, not leaking implementation, etc.
>> That's feeling like the right moment to seize this opportunity. We
>> wouldn't keep the odd location of some of the classes we've already
>> mentioned. And as Cédric says, we could also offer a converter in a way or
>> another to help the migration.
>> People fear transitions like Python 2 to 3 would happen as soon as we
>> break compatibility, but the differences between Python 2 and 3 were much
>> bigger that what we're speaking about here.
>>
>> On Thu, Nov 26, 2015 at 8:49 PM, Cédric Champeau
>> <ce...@gmail.com> wrote:
>>>
>>> Honestly I don't think 1 or 2 is reasonable. The simple example of XML is
>>> enough to show the damage. We won't be able to have a separate XML module.
>>> That defeats the concepts of modules. Also it would lead to bigger jars,
>>> which is something we want to avoid as much as possible. Last but not least,
>>> the more I think about it, the more I think it's the event that we were all
>>> waiting for to finally do the breaking changes we've thought of for years.
>>> Eventually, we could come up with a smoother migration path, by providing an
>>> automatic source converter (remapping packages). It would have the same
>>> advantages as the ones Rémi talked about.
>>>
>>> 2015-11-26 19:18 GMT+01:00 Pascal Schumacher <pa...@gmx.net>:
>>>>
>>>> Thanks for the detailed analysis Cédric. :)
>>>>
>>>> Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
>>>>>
>>>>> So what can we do?
>>>>>
>>>>> 1. the easiest, fastest path, is to kill all modules that we have
>>>>> today, and go with a single, good old, groovy-all jar. We would go years
>>>>> backwards, and it's definitely not something we want to do. We want to have
>>>>> *more* modularization, in particular for Android, where the current split is
>>>>> still too big.
>>>>> 2. refactor modules so that each module has its own set of packages,
>>>>> and hope that we don't end up with a big groovy-all jar. Seems very
>>>>> unlikely.
>>>>> 3. break binary compatibility, move classes around, reorganize stuff.
>>>>
>>>>
>>>> Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
>>>>>
>>>>> I think we should concentrate on solving the package name conflicts in
>>>>> the new module system first... which basically is route 2. I am pretty sure
>>>>> the jdk9 problems won't end there and we need time to solve these problems
>>>>> as well... Of course we could still think about getting rid of the callsite
>>>>> caching part and depend on say JDK7 as minimum version.
>>>>
>>>>
>>>> I agree, we should try option 2 or - if it's nearly the same as option 1
>>>> - take option 1.
>>>>
>>>> I think option 3 is the worst. With our current lack of resources I
>>>> doubt that we could implement enough "killer" features to motivate people to
>>>> update (getting people to update would be difficult anyway).
>>>>
>>>> Cheers,
>>>> Pascal
>>>
>>>
>>
>>
>>
>> --
>> Guillaume Laforge
>> Apache Groovy committer & PMC member
>> Product Ninja & Advocate at Restlet
>>
>> Blog: http://glaforge.appspot.com/
>> Social: @glaforge / Google+
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
2015-11-28 9:12 GMT+01:00 Thibault Kruse <ti...@googlemail.com>:
> Given the funding of groovy changed last year, this situation will show
> what kind of support exists. I know my PRs are growing long beards.
Yeah, a large proportion of your PRs are tricky because they *might*
involve breaking changes. If we choose the breaking change way for Groovy
3, then it's not a problem anymore. Furthermore, I am much in favor of
rewritten modules for groovy console for example. One of the advantages of
going Apache is that we now have much more contributors and committers, I'm
pretty sure we can organize ourselves so that we have "subprojects" and can
work in parallel on topics that we are interested in.
> Maybe work on this could be modularized in smaller milestones, and be
> pursues over kickstarter or GSoC.
>
>
I don't think it's a very sane thing to rely on a kickstarter for an Apache
project, but why not.
> Also I remember another discussion where I put forward the idea of
> reducing the core codebase. Meaning to split out things like groovysh.
>
>
Oh yes. The smaller core, the better.
> It's not ideal maybe, but this might be a 'pick the lesser of two evils'
> situation.
>
>
> On Thursday, November 26, 2015, Guillaume Laforge <gl...@gmail.com>
> wrote:
>
>> I'm also thinking it's the right moment to "fix" things we've done wrong,
>> have a clean separation, not leaking implementation, etc.
>> That's feeling like the right moment to seize this opportunity. We
>> wouldn't keep the odd location of some of the classes we've already
>> mentioned. And as Cédric says, we could also offer a converter in a way or
>> another to help the migration.
>> People fear transitions like Python 2 to 3 would happen as soon as we
>> break compatibility, but the differences between Python 2 and 3 were much
>> bigger that what we're speaking about here.
>>
>> On Thu, Nov 26, 2015 at 8:49 PM, Cédric Champeau <
>> cedric.champeau@gmail.com> wrote:
>>
>>> Honestly I don't think 1 or 2 is reasonable. The simple example of XML
>>> is enough to show the damage. We won't be able to have a separate XML
>>> module. That defeats the concepts of modules. Also it would lead to bigger
>>> jars, which is something we want to avoid as much as possible. Last but not
>>> least, the more I think about it, the more I think it's the event that we
>>> were all waiting for to finally do the breaking changes we've thought of
>>> for years. Eventually, we could come up with a smoother migration path, by
>>> providing an automatic source converter (remapping packages). It would have
>>> the same advantages as the ones Rémi talked about.
>>>
>>> 2015-11-26 19:18 GMT+01:00 Pascal Schumacher <pa...@gmx.net>:
>>>
>>>> Thanks for the detailed analysis Cédric. :)
>>>>
>>>> Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
>>>>
>>>>> So what can we do?
>>>>>
>>>>> 1. the easiest, fastest path, is to kill all modules that we have
>>>>> today, and go with a single, good old, groovy-all jar. We would go years
>>>>> backwards, and it's definitely not something we want to do. We want to have
>>>>> *more* modularization, in particular for Android, where the current split
>>>>> is still too big.
>>>>> 2. refactor modules so that each module has its own set of packages,
>>>>> and hope that we don't end up with a big groovy-all jar. Seems very
>>>>> unlikely.
>>>>> 3. break binary compatibility, move classes around, reorganize stuff.
>>>>>
>>>>
>>>> Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
>>>>
>>>>> I think we should concentrate on solving the package name conflicts in
>>>>> the new module system first... which basically is route 2. I am pretty sure
>>>>> the jdk9 problems won't end there and we need time to solve these problems
>>>>> as well... Of course we could still think about getting rid of the callsite
>>>>> caching part and depend on say JDK7 as minimum version.
>>>>>
>>>>
>>>> I agree, we should try option 2 or - if it's nearly the same as option
>>>> 1 - take option 1.
>>>>
>>>> I think option 3 is the worst. With our current lack of resources I
>>>> doubt that we could implement enough "killer" features to motivate people
>>>> to update (getting people to update would be difficult anyway).
>>>>
>>>> Cheers,
>>>> Pascal
>>>>
>>>
>>>
>>
>>
>> --
>> Guillaume Laforge
>> Apache Groovy committer & PMC member
>> Product Ninja & Advocate at Restlet <http://restlet.com>
>>
>> Blog: http://glaforge.appspot.com/
>> Social: @glaforge <http://twitter.com/glaforge> / Google+
>> <https://plus.google.com/u/0/114130972232398734985/posts>
>>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Thibault Kruse <ti...@googlemail.com>.
Given the funding of groovy changed last year, this situation will show
what kind of support exists. I know my PRs are growing long beards. Maybe
work on this could be modularized in smaller milestones, and be pursues
over kickstarter or GSoC.
Also I remember another discussion where I put forward the idea of reducing
the core codebase. Meaning to split out things like groovysh.
It's not ideal maybe, but this might be a 'pick the lesser of two evils'
situation.
On Thursday, November 26, 2015, Guillaume Laforge <gl...@gmail.com>
wrote:
> I'm also thinking it's the right moment to "fix" things we've done wrong,
> have a clean separation, not leaking implementation, etc.
> That's feeling like the right moment to seize this opportunity. We
> wouldn't keep the odd location of some of the classes we've already
> mentioned. And as Cédric says, we could also offer a converter in a way or
> another to help the migration.
> People fear transitions like Python 2 to 3 would happen as soon as we
> break compatibility, but the differences between Python 2 and 3 were much
> bigger that what we're speaking about here.
>
> On Thu, Nov 26, 2015 at 8:49 PM, Cédric Champeau <
> cedric.champeau@gmail.com
> <javascript:_e(%7B%7D,'cvml','cedric.champeau@gmail.com');>> wrote:
>
>> Honestly I don't think 1 or 2 is reasonable. The simple example of XML is
>> enough to show the damage. We won't be able to have a separate XML module.
>> That defeats the concepts of modules. Also it would lead to bigger jars,
>> which is something we want to avoid as much as possible. Last but not
>> least, the more I think about it, the more I think it's the event that we
>> were all waiting for to finally do the breaking changes we've thought of
>> for years. Eventually, we could come up with a smoother migration path, by
>> providing an automatic source converter (remapping packages). It would have
>> the same advantages as the ones Rémi talked about.
>>
>> 2015-11-26 19:18 GMT+01:00 Pascal Schumacher <pascalschumacher@gmx.net
>> <javascript:_e(%7B%7D,'cvml','pascalschumacher@gmx.net');>>:
>>
>>> Thanks for the detailed analysis Cédric. :)
>>>
>>> Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
>>>
>>>> So what can we do?
>>>>
>>>> 1. the easiest, fastest path, is to kill all modules that we have
>>>> today, and go with a single, good old, groovy-all jar. We would go years
>>>> backwards, and it's definitely not something we want to do. We want to have
>>>> *more* modularization, in particular for Android, where the current split
>>>> is still too big.
>>>> 2. refactor modules so that each module has its own set of packages,
>>>> and hope that we don't end up with a big groovy-all jar. Seems very
>>>> unlikely.
>>>> 3. break binary compatibility, move classes around, reorganize stuff.
>>>>
>>>
>>> Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
>>>
>>>> I think we should concentrate on solving the package name conflicts in
>>>> the new module system first... which basically is route 2. I am pretty sure
>>>> the jdk9 problems won't end there and we need time to solve these problems
>>>> as well... Of course we could still think about getting rid of the callsite
>>>> caching part and depend on say JDK7 as minimum version.
>>>>
>>>
>>> I agree, we should try option 2 or - if it's nearly the same as option 1
>>> - take option 1.
>>>
>>> I think option 3 is the worst. With our current lack of resources I
>>> doubt that we could implement enough "killer" features to motivate people
>>> to update (getting people to update would be difficult anyway).
>>>
>>> Cheers,
>>> Pascal
>>>
>>
>>
>
>
> --
> Guillaume Laforge
> Apache Groovy committer & PMC member
> Product Ninja & Advocate at Restlet <http://restlet.com>
>
> Blog: http://glaforge.appspot.com/
> Social: @glaforge <http://twitter.com/glaforge> / Google+
> <https://plus.google.com/u/0/114130972232398734985/posts>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jorge Franco <gr...@gmail.com>.
Hello! Sorry I'm not a groovy developer, yet!
I think path 3 is the way to go. I want to see groovy on devices, IoT's,
lambda's, containers,... I'm sure is a lot of work, but if we want to go
together with java, we have to do that effort. I feel dirty when a lambda
doesn't compile. I'll help, and sure more people will do the same. Just
define the changes, assign mentors for each big change and let's do this!
Groovy 10 more years please :)
2015-11-26 22:45 GMT+01:00 Jesper Steen Møller <je...@selskabet.org>:
> Hi list
>
> If it’s primarily a question of moving files in modules out into distinct
> package names, how about doing the following:
> 1) Move to a Jigsaw-compatible module split going forward, thus breaking
> compatibility for Jigsaw adopters, and
> 2) Provide a “compatibility” overlay jar containing all the classes with
> old package names for non-jigsaw users?
>
> That way, only people targetting Jigsaw-enabled runtimes will be hit by
> the source imcompatibility.
>
> -Jesper
>
> > On 26. nov. 2015, at 21.29, Jochen Theodorou <bl...@gmx.org> wrote:
> >
> > On 26.11.2015 21:05, Guillaume Laforge wrote:
> >> I'm also thinking it's the right moment to "fix" things we've done
> >> wrong, have a clean separation, not leaking implementation, etc.
> >> That's feeling like the right moment to seize this opportunity. We
> >> wouldn't keep the odd location of some of the classes we've already
> >> mentioned. And as Cédric says, we could also offer a converter in a way
> >> or another to help the migration.
> >> People fear transitions like Python 2 to 3 would happen as soon as we
> >> break compatibility, but the differences between Python 2 and 3 were
> >> much bigger that what we're speaking about here.
> >
> > I think we need a list of the specific cases, then we can talk about the
> seize of the impact.
> >
> > You two know I was all for a big change (MOP2). I am worried about the
> manpower to actually do that change. I was back then already actually and
> did not want to do it all alone.
> >
> > If a source converter can be done the barrier sure is smaller. On the
> other hand Python had https://docs.python.org/2/library/2to3.html
> >
> > bye blackdrag
>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 28.11.2015 10:25, Cédric Champeau wrote:
> Modules can export internals to Groovy, thanks to the concept of
> "automatic modules". However, the automatic modules are inferred from
> the jar name. Which is a weak strategy, but honestly I can't find any
> better solution. So for groovy, since we have multiple flavors, this is
> a problem. Because we have groovy-all, groovy, and all individual
> modules. That's just a nightmare.
>
> BTW, I recommend that everybody interested in this topic takes an hour
> to see this talk from Mark Reinholds:
> https://www.youtube.com/watch?v=UKC0uC7QUkI
ah right, I messed up a little... the unnamed module is for classes
without jar... for example our runtime generated scripts.
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
Modules can export internals to Groovy, thanks to the concept of "automatic
modules". However, the automatic modules are inferred from the jar name.
Which is a weak strategy, but honestly I can't find any better solution. So
for groovy, since we have multiple flavors, this is a problem. Because we
have groovy-all, groovy, and all individual modules. That's just a
nightmare.
BTW, I recommend that everybody interested in this topic takes an hour to
see this talk from Mark Reinholds:
https://www.youtube.com/watch?v=UKC0uC7QUkI
2015-11-28 10:20 GMT+01:00 Jochen Theodorou <bl...@gmx.org>:
> On 27.11.2015 09:24, Cédric Champeau wrote:
>
>>
>>
>> 2015-11-26 23:58 GMT+01:00 Jeff MAURY <jeffmaury@jeffmaury.com
>> <ma...@jeffmaury.com>>:
>>
>> +1 for Jesper proposition with the modification that 2 being
>> groovy-all.jar with binary compatibility but also a Jigsaw module.
>> So we could have:
>> Groovy 3.x: several Jigsaw module refactored
>> Groovy 2.x: same packaging with groovy-all being a stand alone
>> Jigsaw module.
>>
>>
>> I think that's more or less strictly equivalent to the current
>> situation: Groovy 2 "groovy-all" being a jigsaw module can be as simple
>> as exporting all packages. And we don't even have to: just put the jar
>> on classpath and it's done. The problem comes as soon as we want to
>> restrict some things to internal packages.
>>
>
> That's not fully right. Being on the classpath only means we will then be
> in the unnamed module. Other modules have then no way of exporting their
> internals to groovy (well, I think there are ways at runtime, but I am not
> sure). Which means groovy cannot call methods in those packages, which
> means those modules cannot be written in Groovy... at least that is how I
> do understand things.
>
> Today, all the Groovy classes
>> are exposed. Everything belongs, de facto, to the public API.
>>
>
> Well, we did not even really try to prevent this in the past. The mess
> could have been lowered with a more restrictive package named based
> convention.
>
>
> bye blackdrag
>
Re: challenges through Java modules (aka jigsaw)
Posted by Guillaume Laforge <gl...@gmail.com>.
+1, amen :-)
On Fri, Nov 27, 2015 at 9:24 AM, Cédric Champeau <ce...@gmail.com>
wrote:
>
>
> 2015-11-26 23:58 GMT+01:00 Jeff MAURY <je...@jeffmaury.com>:
>
>> +1 for Jesper proposition with the modification that 2 being
>> groovy-all.jar with binary compatibility but also a Jigsaw module.
>> So we could have:
>> Groovy 3.x: several Jigsaw module refactored
>> Groovy 2.x: same packaging with groovy-all being a stand alone Jigsaw
>> module.
>>
>
> I think that's more or less strictly equivalent to the current situation:
> Groovy 2 "groovy-all" being a jigsaw module can be as simple as exporting
> all packages. And we don't even have to: just put the jar on classpath and
> it's done. The problem comes as soon as we want to restrict some things to
> internal packages. Today, all the Groovy classes are exposed. Everything
> belongs, de facto, to the public API. Also remember that we have limited
> resources. Having to maintain multiple jars + jigsaw + android flavors
> while we already have a combination of legacy Groovy + indy Groovy +
> android Groovy, it's just going to be a nightmare for Maven users (of
> course if you use Gradle it's much easier to substitute one transitive
> dependency with another).
>
> My position is that we have so many things to do that it's time to move
> forward. I wasn't in that mindset a few weeks ago, but now I think that we
> should take advantage that Java 9 is going to break everything in any case.
> Also, today, almost nobody use the invokedynamic version because it's too
> complicated to activate, because of transitive dependencies (libraries are
> built with legacy Groovy, but you want to use the indy version). So let's
> simplify everything once for all.
>
>
>>
>> Jeff
>>
>> On Thu, Nov 26, 2015 at 10:45 PM, Jesper Steen Møller <
>> jesper@selskabet.org> wrote:
>>
>>> Hi list
>>>
>>> If it’s primarily a question of moving files in modules out into
>>> distinct package names, how about doing the following:
>>> 1) Move to a Jigsaw-compatible module split going forward, thus breaking
>>> compatibility for Jigsaw adopters, and
>>> 2) Provide a “compatibility” overlay jar containing all the classes with
>>> old package names for non-jigsaw users?
>>>
>>> That way, only people targetting Jigsaw-enabled runtimes will be hit by
>>> the source imcompatibility.
>>>
>>> -Jesper
>>>
>>> > On 26. nov. 2015, at 21.29, Jochen Theodorou <bl...@gmx.org>
>>> wrote:
>>> >
>>> > On 26.11.2015 21:05, Guillaume Laforge wrote:
>>> >> I'm also thinking it's the right moment to "fix" things we've done
>>> >> wrong, have a clean separation, not leaking implementation, etc.
>>> >> That's feeling like the right moment to seize this opportunity. We
>>> >> wouldn't keep the odd location of some of the classes we've already
>>> >> mentioned. And as Cédric says, we could also offer a converter in a
>>> way
>>> >> or another to help the migration.
>>> >> People fear transitions like Python 2 to 3 would happen as soon as we
>>> >> break compatibility, but the differences between Python 2 and 3 were
>>> >> much bigger that what we're speaking about here.
>>> >
>>> > I think we need a list of the specific cases, then we can talk about
>>> the seize of the impact.
>>> >
>>> > You two know I was all for a big change (MOP2). I am worried about the
>>> manpower to actually do that change. I was back then already actually and
>>> did not want to do it all alone.
>>> >
>>> > If a source converter can be done the barrier sure is smaller. On the
>>> other hand Python had https://docs.python.org/2/library/2to3.html
>>> >
>>> > bye blackdrag
>>>
>>>
>>
>>
>> --
>> Jeff MAURY
>>
>>
>> "Legacy code" often differs from its suggested alternative by actually
>> working and scaling.
>> - Bjarne Stroustrup
>>
>> http://www.jeffmaury.com
>> http://riadiscuss.jeffmaury.com
>> http://www.twitter.com/jeffmaury
>>
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC member
Product Ninja & Advocate at Restlet <http://restlet.com>
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 27.11.2015 09:24, Cédric Champeau wrote:
>
>
> 2015-11-26 23:58 GMT+01:00 Jeff MAURY <jeffmaury@jeffmaury.com
> <ma...@jeffmaury.com>>:
>
> +1 for Jesper proposition with the modification that 2 being
> groovy-all.jar with binary compatibility but also a Jigsaw module.
> So we could have:
> Groovy 3.x: several Jigsaw module refactored
> Groovy 2.x: same packaging with groovy-all being a stand alone
> Jigsaw module.
>
>
> I think that's more or less strictly equivalent to the current
> situation: Groovy 2 "groovy-all" being a jigsaw module can be as simple
> as exporting all packages. And we don't even have to: just put the jar
> on classpath and it's done. The problem comes as soon as we want to
> restrict some things to internal packages.
That's not fully right. Being on the classpath only means we will then
be in the unnamed module. Other modules have then no way of exporting
their internals to groovy (well, I think there are ways at runtime, but
I am not sure). Which means groovy cannot call methods in those
packages, which means those modules cannot be written in Groovy... at
least that is how I do understand things.
> Today, all the Groovy classes
> are exposed. Everything belongs, de facto, to the public API.
Well, we did not even really try to prevent this in the past. The mess
could have been lowered with a more restrictive package named based
convention.
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 27.11.2015 19:22, Russel Winder wrote:
[...]
> What do you want GPars to do wrt the Groovy/JDK9 situation?
Our goal after incubation was to get Gpars integrated into Groovy,
right? Which means we will have to solve that as well, of course... Or
is the Gpars situation different now?
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Fri, 2015-11-27 at 09:24 +0100, Cédric Champeau wrote:
>
[…]
> I think that's more or less strictly equivalent to the current
> situation:
> Groovy 2 "groovy-all" being a jigsaw module can be as simple as
> exporting
> all packages. And we don't even have to: just put the jar on
> classpath and
> it's done. The problem comes as soon as we want to restrict some
> things to
> internal packages. Today, all the Groovy classes are exposed.
> Everything
> belongs, de facto, to the public API. Also remember that we have
> limited
> resources. Having to maintain multiple jars + jigsaw + android
> flavors
> while we already have a combination of legacy Groovy + indy Groovy +
> android Groovy, it's just going to be a nightmare for Maven users (of
> course if you use Gradle it's much easier to substitute one
> transitive
> dependency with another).
>
> My position is that we have so many things to do that it's time to
> move
> forward. I wasn't in that mindset a few weeks ago, but now I think
> that we
> should take advantage that Java 9 is going to break everything in any
> case.
> Also, today, almost nobody use the invokedynamic version because it's
> too
> complicated to activate, because of transitive dependencies
> (libraries are
> built with legacy Groovy, but you want to use the indy version). So
> let's
> simplify everything once for all.
The indy artefact is the only one I use and only on JDK9. Sadly use of
invokedynamic is not the default.
What do you want GPars to do wrt the Groovy/JDK9 situation?
Sadly, currently the JDK8 version of GPars is in a mess, so the JDK7
version is all that works. I suspect 4 people in a days work could fix
almost all of it, but will such a hackathon happen at Groovy and Grails
eXchange 2015?
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
2015-11-26 23:58 GMT+01:00 Jeff MAURY <je...@jeffmaury.com>:
> +1 for Jesper proposition with the modification that 2 being
> groovy-all.jar with binary compatibility but also a Jigsaw module.
> So we could have:
> Groovy 3.x: several Jigsaw module refactored
> Groovy 2.x: same packaging with groovy-all being a stand alone Jigsaw
> module.
>
I think that's more or less strictly equivalent to the current situation:
Groovy 2 "groovy-all" being a jigsaw module can be as simple as exporting
all packages. And we don't even have to: just put the jar on classpath and
it's done. The problem comes as soon as we want to restrict some things to
internal packages. Today, all the Groovy classes are exposed. Everything
belongs, de facto, to the public API. Also remember that we have limited
resources. Having to maintain multiple jars + jigsaw + android flavors
while we already have a combination of legacy Groovy + indy Groovy +
android Groovy, it's just going to be a nightmare for Maven users (of
course if you use Gradle it's much easier to substitute one transitive
dependency with another).
My position is that we have so many things to do that it's time to move
forward. I wasn't in that mindset a few weeks ago, but now I think that we
should take advantage that Java 9 is going to break everything in any case.
Also, today, almost nobody use the invokedynamic version because it's too
complicated to activate, because of transitive dependencies (libraries are
built with legacy Groovy, but you want to use the indy version). So let's
simplify everything once for all.
>
> Jeff
>
> On Thu, Nov 26, 2015 at 10:45 PM, Jesper Steen Møller <
> jesper@selskabet.org> wrote:
>
>> Hi list
>>
>> If it’s primarily a question of moving files in modules out into distinct
>> package names, how about doing the following:
>> 1) Move to a Jigsaw-compatible module split going forward, thus breaking
>> compatibility for Jigsaw adopters, and
>> 2) Provide a “compatibility” overlay jar containing all the classes with
>> old package names for non-jigsaw users?
>>
>> That way, only people targetting Jigsaw-enabled runtimes will be hit by
>> the source imcompatibility.
>>
>> -Jesper
>>
>> > On 26. nov. 2015, at 21.29, Jochen Theodorou <bl...@gmx.org> wrote:
>> >
>> > On 26.11.2015 21:05, Guillaume Laforge wrote:
>> >> I'm also thinking it's the right moment to "fix" things we've done
>> >> wrong, have a clean separation, not leaking implementation, etc.
>> >> That's feeling like the right moment to seize this opportunity. We
>> >> wouldn't keep the odd location of some of the classes we've already
>> >> mentioned. And as Cédric says, we could also offer a converter in a way
>> >> or another to help the migration.
>> >> People fear transitions like Python 2 to 3 would happen as soon as we
>> >> break compatibility, but the differences between Python 2 and 3 were
>> >> much bigger that what we're speaking about here.
>> >
>> > I think we need a list of the specific cases, then we can talk about
>> the seize of the impact.
>> >
>> > You two know I was all for a big change (MOP2). I am worried about the
>> manpower to actually do that change. I was back then already actually and
>> did not want to do it all alone.
>> >
>> > If a source converter can be done the barrier sure is smaller. On the
>> other hand Python had https://docs.python.org/2/library/2to3.html
>> >
>> > bye blackdrag
>>
>>
>
>
> --
> Jeff MAURY
>
>
> "Legacy code" often differs from its suggested alternative by actually
> working and scaling.
> - Bjarne Stroustrup
>
> http://www.jeffmaury.com
> http://riadiscuss.jeffmaury.com
> http://www.twitter.com/jeffmaury
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jeff MAURY <je...@jeffmaury.com>.
+1 for Jesper proposition with the modification that 2 being groovy-all.jar
with binary compatibility but also a Jigsaw module.
So we could have:
Groovy 3.x: several Jigsaw module refactored
Groovy 2.x: same packaging with groovy-all being a stand alone Jigsaw
module.
Jeff
On Thu, Nov 26, 2015 at 10:45 PM, Jesper Steen Møller <je...@selskabet.org>
wrote:
> Hi list
>
> If it’s primarily a question of moving files in modules out into distinct
> package names, how about doing the following:
> 1) Move to a Jigsaw-compatible module split going forward, thus breaking
> compatibility for Jigsaw adopters, and
> 2) Provide a “compatibility” overlay jar containing all the classes with
> old package names for non-jigsaw users?
>
> That way, only people targetting Jigsaw-enabled runtimes will be hit by
> the source imcompatibility.
>
> -Jesper
>
> > On 26. nov. 2015, at 21.29, Jochen Theodorou <bl...@gmx.org> wrote:
> >
> > On 26.11.2015 21:05, Guillaume Laforge wrote:
> >> I'm also thinking it's the right moment to "fix" things we've done
> >> wrong, have a clean separation, not leaking implementation, etc.
> >> That's feeling like the right moment to seize this opportunity. We
> >> wouldn't keep the odd location of some of the classes we've already
> >> mentioned. And as Cédric says, we could also offer a converter in a way
> >> or another to help the migration.
> >> People fear transitions like Python 2 to 3 would happen as soon as we
> >> break compatibility, but the differences between Python 2 and 3 were
> >> much bigger that what we're speaking about here.
> >
> > I think we need a list of the specific cases, then we can talk about the
> seize of the impact.
> >
> > You two know I was all for a big change (MOP2). I am worried about the
> manpower to actually do that change. I was back then already actually and
> did not want to do it all alone.
> >
> > If a source converter can be done the barrier sure is smaller. On the
> other hand Python had https://docs.python.org/2/library/2to3.html
> >
> > bye blackdrag
>
>
--
Jeff MAURY
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: challenges through Java modules (aka jigsaw)
Posted by Jesper Steen Møller <je...@selskabet.org>.
Hi list
If it’s primarily a question of moving files in modules out into distinct package names, how about doing the following:
1) Move to a Jigsaw-compatible module split going forward, thus breaking compatibility for Jigsaw adopters, and
2) Provide a “compatibility” overlay jar containing all the classes with old package names for non-jigsaw users?
That way, only people targetting Jigsaw-enabled runtimes will be hit by the source imcompatibility.
-Jesper
> On 26. nov. 2015, at 21.29, Jochen Theodorou <bl...@gmx.org> wrote:
>
> On 26.11.2015 21:05, Guillaume Laforge wrote:
>> I'm also thinking it's the right moment to "fix" things we've done
>> wrong, have a clean separation, not leaking implementation, etc.
>> That's feeling like the right moment to seize this opportunity. We
>> wouldn't keep the odd location of some of the classes we've already
>> mentioned. And as Cédric says, we could also offer a converter in a way
>> or another to help the migration.
>> People fear transitions like Python 2 to 3 would happen as soon as we
>> break compatibility, but the differences between Python 2 and 3 were
>> much bigger that what we're speaking about here.
>
> I think we need a list of the specific cases, then we can talk about the seize of the impact.
>
> You two know I was all for a big change (MOP2). I am worried about the manpower to actually do that change. I was back then already actually and did not want to do it all alone.
>
> If a source converter can be done the barrier sure is smaller. On the other hand Python had https://docs.python.org/2/library/2to3.html
>
> bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 26.11.2015 21:05, Guillaume Laforge wrote:
> I'm also thinking it's the right moment to "fix" things we've done
> wrong, have a clean separation, not leaking implementation, etc.
> That's feeling like the right moment to seize this opportunity. We
> wouldn't keep the odd location of some of the classes we've already
> mentioned. And as Cédric says, we could also offer a converter in a way
> or another to help the migration.
> People fear transitions like Python 2 to 3 would happen as soon as we
> break compatibility, but the differences between Python 2 and 3 were
> much bigger that what we're speaking about here.
I think we need a list of the specific cases, then we can talk about the
seize of the impact.
You two know I was all for a big change (MOP2). I am worried about the
manpower to actually do that change. I was back then already actually
and did not want to do it all alone.
If a source converter can be done the barrier sure is smaller. On the
other hand Python had https://docs.python.org/2/library/2to3.html
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Guillaume Laforge <gl...@gmail.com>.
I'm also thinking it's the right moment to "fix" things we've done wrong,
have a clean separation, not leaking implementation, etc.
That's feeling like the right moment to seize this opportunity. We wouldn't
keep the odd location of some of the classes we've already mentioned. And
as Cédric says, we could also offer a converter in a way or another to help
the migration.
People fear transitions like Python 2 to 3 would happen as soon as we break
compatibility, but the differences between Python 2 and 3 were much bigger
that what we're speaking about here.
On Thu, Nov 26, 2015 at 8:49 PM, Cédric Champeau <ce...@gmail.com>
wrote:
> Honestly I don't think 1 or 2 is reasonable. The simple example of XML is
> enough to show the damage. We won't be able to have a separate XML module.
> That defeats the concepts of modules. Also it would lead to bigger jars,
> which is something we want to avoid as much as possible. Last but not
> least, the more I think about it, the more I think it's the event that we
> were all waiting for to finally do the breaking changes we've thought of
> for years. Eventually, we could come up with a smoother migration path, by
> providing an automatic source converter (remapping packages). It would have
> the same advantages as the ones Rémi talked about.
>
> 2015-11-26 19:18 GMT+01:00 Pascal Schumacher <pa...@gmx.net>:
>
>> Thanks for the detailed analysis Cédric. :)
>>
>> Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
>>
>>> So what can we do?
>>>
>>> 1. the easiest, fastest path, is to kill all modules that we have today,
>>> and go with a single, good old, groovy-all jar. We would go years
>>> backwards, and it's definitely not something we want to do. We want to have
>>> *more* modularization, in particular for Android, where the current split
>>> is still too big.
>>> 2. refactor modules so that each module has its own set of packages, and
>>> hope that we don't end up with a big groovy-all jar. Seems very unlikely.
>>> 3. break binary compatibility, move classes around, reorganize stuff.
>>>
>>
>> Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
>>
>>> I think we should concentrate on solving the package name conflicts in
>>> the new module system first... which basically is route 2. I am pretty sure
>>> the jdk9 problems won't end there and we need time to solve these problems
>>> as well... Of course we could still think about getting rid of the callsite
>>> caching part and depend on say JDK7 as minimum version.
>>>
>>
>> I agree, we should try option 2 or - if it's nearly the same as option 1
>> - take option 1.
>>
>> I think option 3 is the worst. With our current lack of resources I doubt
>> that we could implement enough "killer" features to motivate people to
>> update (getting people to update would be difficult anyway).
>>
>> Cheers,
>> Pascal
>>
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC member
Product Ninja & Advocate at Restlet <http://restlet.com>
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
Honestly I don't think 1 or 2 is reasonable. The simple example of XML is
enough to show the damage. We won't be able to have a separate XML module.
That defeats the concepts of modules. Also it would lead to bigger jars,
which is something we want to avoid as much as possible. Last but not
least, the more I think about it, the more I think it's the event that we
were all waiting for to finally do the breaking changes we've thought of
for years. Eventually, we could come up with a smoother migration path, by
providing an automatic source converter (remapping packages). It would have
the same advantages as the ones Rémi talked about.
2015-11-26 19:18 GMT+01:00 Pascal Schumacher <pa...@gmx.net>:
> Thanks for the detailed analysis Cédric. :)
>
> Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
>
>> So what can we do?
>>
>> 1. the easiest, fastest path, is to kill all modules that we have today,
>> and go with a single, good old, groovy-all jar. We would go years
>> backwards, and it's definitely not something we want to do. We want to have
>> *more* modularization, in particular for Android, where the current split
>> is still too big.
>> 2. refactor modules so that each module has its own set of packages, and
>> hope that we don't end up with a big groovy-all jar. Seems very unlikely.
>> 3. break binary compatibility, move classes around, reorganize stuff.
>>
>
> Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
>
>> I think we should concentrate on solving the package name conflicts in
>> the new module system first... which basically is route 2. I am pretty sure
>> the jdk9 problems won't end there and we need time to solve these problems
>> as well... Of course we could still think about getting rid of the callsite
>> caching part and depend on say JDK7 as minimum version.
>>
>
> I agree, we should try option 2 or - if it's nearly the same as option 1 -
> take option 1.
>
> I think option 3 is the worst. With our current lack of resources I doubt
> that we could implement enough "killer" features to motivate people to
> update (getting people to update would be difficult anyway).
>
> Cheers,
> Pascal
>
Re: challenges through Java modules (aka jigsaw)
Posted by Pascal Schumacher <pa...@gmx.net>.
Thanks for the detailed analysis Cédric. :)
Am 26.11.2015 um 13:10 schrieb Cédric Champeau:
> So what can we do?
>
> 1. the easiest, fastest path, is to kill all modules that we have
> today, and go with a single, good old, groovy-all jar. We would go
> years backwards, and it's definitely not something we want to do. We
> want to have *more* modularization, in particular for Android, where
> the current split is still too big.
> 2. refactor modules so that each module has its own set of packages,
> and hope that we don't end up with a big groovy-all jar. Seems very
> unlikely.
> 3. break binary compatibility, move classes around, reorganize stuff.
Am 26.11.2015 um 14:16 schrieb Jochen Theodorou:
> I think we should concentrate on solving the package name conflicts in
> the new module system first... which basically is route 2. I am pretty
> sure the jdk9 problems won't end there and we need time to solve these
> problems as well... Of course we could still think about getting rid
> of the callsite caching part and depend on say JDK7 as minimum version.
I agree, we should try option 2 or - if it's nearly the same as option 1
- take option 1.
I think option 3 is the worst. With our current lack of resources I
doubt that we could implement enough "killer" features to motivate
people to update (getting people to update would be difficult anyway).
Cheers,
Pascal
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 26.11.2015 13:10, Cédric Champeau wrote:
> So I'm taking advantage of a bit of free time to write a quick summary
> of the situation and what we will have to decide. Basically, we have a
> serious problem (but we will not be the only ones).
>
> When you declare a Jigsaw module, one has to declare the list of
> packages that it exports (and sub-packages are not exported). Those
> packages are the ones that can be consumed from other Jigsaw modules. It
> therefore implements strong encapsulation, but making it impossible to
> reference an internal class of a module, both at compile time and
> runtime. So far so good. The problem is that the list of packages that
> belong to a module is exclusive to that module. So if module A exports
> "foo.bar", another module cannot do the same, or it will fail whenever
> the 2 modules are loaded at the same time. Worse, *internal packages*
> are also exclusive, so even if you don't export a package, two modules
> containing the same internal packages will *also* fail.
Examples of these fails (copied from a mail to the jigsaw list I did):
* groovy-console has a conflict in groovy.ui
* groovy-ant has a conflict in groovy.util
* groovy-docgenerator in org.codehaus.groovy.tools
* groovy-nio in org.codehaus.groovy.runtime
* groovy-sql in org.codehaus.groovy.runtime
* groovy-swing in org.codehaus.groovy.runtime
* groovy-test in goovy.lang and groovy.util
* groovy-xml in org.codehaus.groovy.runtime and goovy.util
The imho biggest problem is GroovyTestCase being in groovy-test, but
also in groovy.util
[...]
> So what can we do?
>
> 1. the easiest, fastest path, is to kill all modules that we have today,
> and go with a single, good old, groovy-all jar. We would go years
> backwards, and it's definitely not something we want to do. We want to
> have *more* modularization, in particular for Android, where the current
> split is still too big.
I was mentioning the split of runtime and compiler on the jigsaw list.
The suggestion was basically to make a service provider based logic...
But frankly, the android version does not have to use modules at all,
does it?
> 2. refactor modules so that each module has its own set of packages, and
> hope that we don't end up with a big groovy-all jar. Seems very unlikely.
I think the conflicts in org.codehaus.groovy.runtime can be solved.
groovy.util and groovy.lang conflicts are the really problematic ones...
and just giving them their own packages won't prevent way (3). A variant
of (2) is to deprecate all non-solvable conflicting modules and provide
alternatives. That would hit mostly groovy-test and groovy-xml, and
maybe require a conceptual change... we could for example go more the
junit4 route and not require extending GroovyTestCase anymore, but
instead have an annotation.. which then would of course not be in
groovy.util or groovy.lang. Still a huge change of course.
> 3. break binary compatibility, move classes around, reorganize stuff.
>
> Despite my worst nightmare of fragmenting the community (that is what
> happened for all languages that tried to introduce breaking changes), I
> think we don't have the choice and we have to go route 3. And if we do
> so, I would take advantage of this fact to:
> 1. reorganize our code base even further to separate everything into
> their own modules and make them as minimal as possible
> 2. migrate to Java 8 as the minimum JDK version, which would allow us to
> drop support of the old call site caching, get a new MOP, and possibly
> removing everything that annoys us in terms of behavior which is kept
> for the sake of compatibility
We will have to think of milestones for this.... and probably many of them
> It's a long road, and it is pretty obvious that we won't be able to be
> ready for Jigsaw launch (october 2016), but it we don't decide today
> what we want to do, we will soon be obsolete, because nobody will be
> able to use Groovy with Jigsaw.
Not entirely right imho. You can then still use Groovy, but not as a
real module and probably only groovy-all and with nothing written in
Groovy as module. That will of course not help Gradle or Grails.
> Last but not least, as part of the Gradle team, I'd like to take
> advantage of this to rework our build to make it more consistent,
> faster, and aimed towards the future. In next January, we will start
> promoting a smooth migration path to Jigsaw using the Java software
> model. I think Groovy is still too big to migrate now to the java
> software model, but at least, we can start investigating. It is crucial
> to us because Gradle uses Groovy internally, so if Gradle doesn't run on
> Jigsaw, we will have a medium term problem (medium, because we could
> still run on JDK 8 but target JDK 9).
>
> Let the fun begin!
I think a new MOP is out of scope for this. As much as I would like to
have a better MOP, which for a change can be really specified in a
document and is not as overly complex as the current one, we have to be
realistic. And a really new MOP (not one simply tweaked a bit and still
breaking lot's of things) requires imho more than a man year to
implement and migrate the codebase. I don't see that kind of resources
available without more than one fulltime developer. And we don't have
even one. I summed up some ideas for a slower migration path to a new
MOP in
http://blackdragsview.blogspot.ch/2015/03/thoughts-about-new-meta-class-system.html
But well... only some ideas.
I think we should concentrate on solving the package name conflicts in
the new module system first... which basically is route 2. I am pretty
sure the jdk9 problems won't end there and we need time to solve these
problems as well... Of course we could still think about getting rid of
the callsite caching part and depend on say JDK7 as minimum version.
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 27.11.2015 19:06, Russel Winder wrote:
[...]
> I suspect it was some Californian crack
> consuming marketeer who came up with the idea to try and create a new
> branding. That person needs putting down so as to save humanity from
> crimes against language.
It is funny I did read that as a Californian crack, which consumed a
marketing guy ;) lol
bye blackdrag
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Sat, 2015-11-28 at 10:26 +0100, Cédric Champeau wrote:
> Sorry for the misused terminology. But one has to be aware that we're
> not
> all native speakers :)
The root of the problem is the supposed native English speakers who
invented the meaning and usage to seem trendy and different and then
used their position to make it seem normal usage. So when people such
as yourself use it, you are doing so because of the original. The fault
is theirs, not yours.
I just ask that we do not continue their improper use of English.
(I am sure the same sort of things happen in French, German, Spanish,…
there are "marketing dudes" bound to every language.)
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
Sorry for the misused terminology. But one has to be aware that we're not
all native speakers :)
2015-11-27 19:06 GMT+01:00 Russel Winder <ru...@winder.org.uk>:
> On Thu, 2015-11-26 at 13:10 +0100, Cédric Champeau wrote:
> […]
> > ones that can be consumed from other Jigsaw modules. It therefore
> […]
>
> s/consumed from/used by/
>
> I have no idea when this "consumed" thing came about in API speak, but
> it is fundamentally wrong. I suspect it was some Californian crack
> consuming marketeer who came up with the idea to try and create a new
> branding. That person needs putting down so as to save humanity from
> crimes against language.
>
> --
> Russel.
>
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip:
> sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Thu, 2015-11-26 at 13:10 +0100, Cédric Champeau wrote:
[…]
> ones that can be consumed from other Jigsaw modules. It therefore
[…]
s/consumed from/used by/
I have no idea when this "consumed" thing came about in API speak, but
it is fundamentally wrong. I suspect it was some Californian crack
consuming marketeer who came up with the idea to try and create a new
branding. That person needs putting down so as to save humanity from
crimes against language.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 26.11.2015 13:58, Remi Forax wrote:
>
>
> ------------------------------------------------------------------------
>
> *De: *"Cédric Champeau" <ce...@gmail.com>
> *À: *dev@groovy.incubator.apache.org
> *Envoyé: *Jeudi 26 Novembre 2015 13:49:18
> *Objet: *Re: challenges through Java modules (aka jigsaw)
>
>
>
> You can have breaking changes in the way the classes are seen
> from Java (and jigsaw) without having breaking changes when you
> use these classes from Groovy.
> By example, you can introduce a way to create type aliases to
> Groovy, given that, you can move Java classes around and keep
> the same references in the Groovy code.
>
> This would work only if a library is recompiled against the latest
> Groovy version (and even more true if you use @CompileStatic),
> whereas the idea of backwards compatibility is *not* to recompile
> everything. That is to say, take Groovy N+1 and it is capable of
> running classes compiled with Groovy N.
>
>
> The translation of types can appear at runtime if the type aliases are
> known by the groovy class loader.
> I think you only need re-compilation if you use @CompileStatic.
You mean loading of foo.bar.X is requested, but instead the classloader
gives out bar.foo.Y? I guess that works even under classloader
constraints....
Problem is that there are different environments to consider.
Precompiled and executed from Java for example means to have an
environment without a classloader under our control. Your suggestion
would not work there. If the script is not precompiled, then there is
always a GroovyClassLoader involved we can use, yes.... Even though
RootLoader would be the better place imho, but RootLoader too is not
always there. But maybe a tool operating on the bytecode can be used for
the precompiled cases. So I agree with Remi and we should not exclude
this idea prematurely.
>
> Also about groovy-all, you can create a module groovy-all that
> will require all other modules, no ?
>
> Yes, but that's not the purpose of this module today. It's more for
> guys who are too lazy to precisely determine all their Groovy module
> dependencies and prefer to use groovy-all instead of groovy + "find
> all modules that I use". Note that the simplification that I give is
> recent, because a few releases ago, groovy-all contained repackaged
> ASM classes, whereas groovy.jar did not. Now both have repackaged
> classes, making it much more consistent and even less useful to use
> groovy-all.
I really see even less problems with groovy-all compared to the non-all
groovy jar for normal java. Android is different... but that could do
with a special jar
> if it's only packaging, you just have to create a module-info.class that
> will concatenate all the module-info.class of all other modules, you may
> need ASM for that :)
actually gradle may even have that, or will have that (using asm then of
course).
bye Jochen
Re: challenges through Java modules (aka jigsaw)
Posted by Remi Forax <fo...@univ-mlv.fr>.
----- Mail original -----
> De: "Cédric Champeau" <ce...@gmail.com>
> À: dev@groovy.incubator.apache.org
> Envoyé: Jeudi 26 Novembre 2015 13:49:18
> Objet: Re: challenges through Java modules (aka jigsaw)
> > You can have breaking changes in the way the classes are seen from Java
> > (and
> > jigsaw) without having breaking changes when you use these classes from
> > Groovy.
>
> > By example, you can introduce a way to create type aliases to Groovy, given
> > that, you can move Java classes around and keep the same references in the
> > Groovy code.
>
> This would work only if a library is recompiled against the latest Groovy
> version (and even more true if you use @CompileStatic), whereas the idea of
> backwards compatibility is *not* to recompile everything. That is to say,
> take Groovy N+1 and it is capable of running classes compiled with Groovy N.
The translation of types can appear at runtime if the type aliases are known by the groovy class loader.
I think you only need re-compilation if you use @CompileStatic.
> > Also about groovy-all, you can create a module groovy-all that will require
> > all other modules, no ?
>
> Yes, but that's not the purpose of this module today. It's more for guys who
> are too lazy to precisely determine all their Groovy module dependencies and
> prefer to use groovy-all instead of groovy + "find all modules that I use".
> Note that the simplification that I give is recent, because a few releases
> ago, groovy-all contained repackaged ASM classes, whereas groovy.jar did
> not. Now both have repackaged classes, making it much more consistent and
> even less useful to use groovy-all.
if it's only packaging, you just have to create a module-info.class that will concatenate all the module-info.class of all other modules, you may need ASM for that :)
Rémi
> > regards,
>
> > Rémi
>
> > > De: "Cédric Champeau" < cedric.champeau@gmail.com >
> >
>
> > > À: dev@groovy.incubator.apache.org
> >
>
> > > Envoyé: Jeudi 26 Novembre 2015 13:10:11
> >
>
> > > Objet: Re: challenges through Java modules (aka jigsaw)
> >
>
> > > So I'm taking advantage of a bit of free time to write a quick summary of
> > > the
> > > situation and what we will have to decide. Basically, we have a serious
> > > problem (but we will not be the only ones).
> >
>
> > > When you declare a Jigsaw module, one has to declare the list of packages
> > > that it exports (and sub-packages are not exported). Those packages are
> > > the
> > > ones that can be consumed from other Jigsaw modules. It therefore
> > > implements
> > > strong encapsulation, but making it impossible to reference an internal
> > > class of a module, both at compile time and runtime. So far so good. The
> > > problem is that the list of packages that belong to a module is exclusive
> > > to
> > > that module. So if module A exports "foo.bar", another module cannot do
> > > the
> > > same, or it will fail whenever the 2 modules are loaded at the same time.
> > > Worse, *internal packages* are also exclusive, so even if you don't
> > > export
> > > a
> > > package, two modules containing the same internal packages will *also*
> > > fail.
> >
>
> > > Why is it a problem for Groovy? It is a problem because we have a very
> > > long
> > > history. And part of this history makes that modularization was
> > > introduced
> > > pretty recently, in the 2.0 release. Before, Groovy only had a single
> > > module. Now Groovy provides multiple modules, but also a "groovy-all" jar
> > > that contains all the classes from all modules. At this point, we can
> > > already see that this won't be possible anymore: groovy-all must die, or
> > > we
> > > must not support modules (sigh). But the problem is even worse. Since the
> > > split into modules was done late and that we wanted to preserve backwards
> > > compatibility, we decided not to change the packages of classes extracted
> > > into their own modules. So for example, XmlParser is found in groovy.util
> > > (not groovy.util.xml) in the groovy-xml module, but we also have lots of
> > > classes that live into groovy.xml into the "groovy" core module. So in
> > > short, our modules are no fit for Jigsaw: we must come back to the very
> > > bad
> > > state of having everything in a single module.
> >
>
> > > Of course, that's not a satisfactory solution, and of course, there's
> > > nothing
> > > we can do to influence the decision not to make such a strong
> > > requirement:
> > > there are technical reasons behind this choice.
> >
>
> > > So what can we do?
> >
>
> > > 1. the easiest, fastest path, is to kill all modules that we have today,
> > > and
> > > go with a single, good old, groovy-all jar. We would go years backwards,
> > > and
> > > it's definitely not something we want to do. We want to have *more*
> > > modularization, in particular for Android, where the current split is
> > > still
> > > too big.
> >
>
> > > 2. refactor modules so that each module has its own set of packages, and
> > > hope
> > > that we don't end up with a big groovy-all jar. Seems very unlikely.
> >
>
> > > 3. break binary compatibility, move classes around, reorganize stuff.
> >
>
> > > Despite my worst nightmare of fragmenting the community (that is what
> > > happened for all languages that tried to introduce breaking changes), I
> > > think we don't have the choice and we have to go route 3. And if we do
> > > so,
> > > I
> > > would take advantage of this fact to:
> >
>
> > > 1. reorganize our code base even further to separate everything into
> > > their
> > > own modules and make them as minimal as possible
> >
>
> > > 2. migrate to Java 8 as the minimum JDK version, which would allow us to
> > > drop
> > > support of the old call site caching, get a new MOP, and possibly
> > > removing
> > > everything that annoys us in terms of behavior which is kept for the sake
> > > of
> > > compatibility
> >
>
> > > It's a long road, and it is pretty obvious that we won't be able to be
> > > ready
> > > for Jigsaw launch (october 2016), but it we don't decide today what we
> > > want
> > > to do, we will soon be obsolete, because nobody will be able to use
> > > Groovy
> > > with Jigsaw.
> >
>
> > > Last but not least, as part of the Gradle team, I'd like to take
> > > advantage
> > > of
> > > this to rework our build to make it more consistent, faster, and aimed
> > > towards the future. In next January, we will start promoting a smooth
> > > migration path to Jigsaw using the Java software model. I think Groovy is
> > > still too big to migrate now to the java software model, but at least, we
> > > can start investigating. It is crucial to us because Gradle uses Groovy
> > > internally, so if Gradle doesn't run on Jigsaw, we will have a medium
> > > term
> > > problem (medium, because we could still run on JDK 8 but target JDK 9).
> >
>
> > > Let the fun begin!
> >
>
> > > 2015-11-15 17:07 GMT+01:00 Jochen Theodorou < blackdrag@gmx.org > :
> >
>
> > > > On 15.11.2015 12:11, Cédric Champeau wrote:
> > >
> >
>
> > > > > Hi guys,
> > > >
> > >
> >
>
> > > > > After spending some days at Devoxx, I had a few chats about Jigsaw
> > > > > and
> > > >
> > >
> >
>
> > > > > the implications on Groovy. I will send an email summarizing them and
> > > > > I
> > > >
> > >
> >
>
> > > > > think we need to come up with a plan.
> > > >
> > >
> >
>
> > > > I was saying that, yes. Seems nobody wanted to believe me
> > >
> >
>
> > > > > I'm not sure JIRA is the best way
> > > >
> > >
> >
>
> > > > > to track this, but in short, we have serious issues coming, and
> > > > > breaking
> > > >
> > >
> >
>
> > > > > changes. Maybe it's a chance for us to make all the breaking changes
> > > > > we
> > > >
> > >
> >
>
> > > > > thought for so long.
> > > >
> > >
> >
>
> > > > I doubt it, we don't have the manpower, but we can do a bit of course
> > >
> >
>
> > > > bye Jochen
> > >
> >
>
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
>
>
>
> You can have breaking changes in the way the classes are seen from Java
> (and jigsaw) without having breaking changes when you use these classes
> from Groovy.
> By example, you can introduce a way to create type aliases to Groovy,
> given that, you can move Java classes around and keep the same references
> in the Groovy code.
>
> This would work only if a library is recompiled against the latest Groovy
version (and even more true if you use @CompileStatic), whereas the idea of
backwards compatibility is *not* to recompile everything. That is to say,
take Groovy N+1 and it is capable of running classes compiled with Groovy N.
> Also about groovy-all, you can create a module groovy-all that will
> require all other modules, no ?
>
Yes, but that's not the purpose of this module today. It's more for guys
who are too lazy to precisely determine all their Groovy module
dependencies and prefer to use groovy-all instead of groovy + "find all
modules that I use". Note that the simplification that I give is recent,
because a few releases ago, groovy-all contained repackaged ASM classes,
whereas groovy.jar did not. Now both have repackaged classes, making it
much more consistent and even less useful to use groovy-all.
>
> regards,
> Rémi
>
> ------------------------------
>
> *De: *"Cédric Champeau" <ce...@gmail.com>
> *À: *dev@groovy.incubator.apache.org
> *Envoyé: *Jeudi 26 Novembre 2015 13:10:11
> *Objet: *Re: challenges through Java modules (aka jigsaw)
>
>
> So I'm taking advantage of a bit of free time to write a quick summary of
> the situation and what we will have to decide. Basically, we have a serious
> problem (but we will not be the only ones).
>
> When you declare a Jigsaw module, one has to declare the list of packages
> that it exports (and sub-packages are not exported). Those packages are the
> ones that can be consumed from other Jigsaw modules. It therefore
> implements strong encapsulation, but making it impossible to reference an
> internal class of a module, both at compile time and runtime. So far so
> good. The problem is that the list of packages that belong to a module is
> exclusive to that module. So if module A exports "foo.bar", another module
> cannot do the same, or it will fail whenever the 2 modules are loaded at
> the same time. Worse, *internal packages* are also exclusive, so even if
> you don't export a package, two modules containing the same internal
> packages will *also* fail.
>
> Why is it a problem for Groovy? It is a problem because we have a very
> long history. And part of this history makes that modularization was
> introduced pretty recently, in the 2.0 release. Before, Groovy only had a
> single module. Now Groovy provides multiple modules, but also a
> "groovy-all" jar that contains all the classes from all modules. At this
> point, we can already see that this won't be possible anymore: groovy-all
> must die, or we must not support modules (sigh). But the problem is even
> worse. Since the split into modules was done late and that we wanted to
> preserve backwards compatibility, we decided not to change the packages of
> classes extracted into their own modules. So for example, XmlParser is
> found in groovy.util (not groovy.util.xml) in the groovy-xml module, but we
> also have lots of classes that live into groovy.xml into the "groovy" core
> module. So in short, our modules are no fit for Jigsaw: we must come back
> to the very bad state of having everything in a single module.
>
> Of course, that's not a satisfactory solution, and of course, there's
> nothing we can do to influence the decision not to make such a strong
> requirement: there are technical reasons behind this choice.
>
> So what can we do?
>
> 1. the easiest, fastest path, is to kill all modules that we have today,
> and go with a single, good old, groovy-all jar. We would go years
> backwards, and it's definitely not something we want to do. We want to have
> *more* modularization, in particular for Android, where the current split
> is still too big.
> 2. refactor modules so that each module has its own set of packages, and
> hope that we don't end up with a big groovy-all jar. Seems very unlikely.
> 3. break binary compatibility, move classes around, reorganize stuff.
>
> Despite my worst nightmare of fragmenting the community (that is what
> happened for all languages that tried to introduce breaking changes), I
> think we don't have the choice and we have to go route 3. And if we do so,
> I would take advantage of this fact to:
> 1. reorganize our code base even further to separate everything into their
> own modules and make them as minimal as possible
> 2. migrate to Java 8 as the minimum JDK version, which would allow us to
> drop support of the old call site caching, get a new MOP, and possibly
> removing everything that annoys us in terms of behavior which is kept for
> the sake of compatibility
>
> It's a long road, and it is pretty obvious that we won't be able to be
> ready for Jigsaw launch (october 2016), but it we don't decide today what
> we want to do, we will soon be obsolete, because nobody will be able to use
> Groovy with Jigsaw.
>
> Last but not least, as part of the Gradle team, I'd like to take advantage
> of this to rework our build to make it more consistent, faster, and aimed
> towards the future. In next January, we will start promoting a smooth
> migration path to Jigsaw using the Java software model. I think Groovy is
> still too big to migrate now to the java software model, but at least, we
> can start investigating. It is crucial to us because Gradle uses Groovy
> internally, so if Gradle doesn't run on Jigsaw, we will have a medium term
> problem (medium, because we could still run on JDK 8 but target JDK 9).
>
> Let the fun begin!
>
> 2015-11-15 17:07 GMT+01:00 Jochen Theodorou <bl...@gmx.org>:
>
>> On 15.11.2015 12:11, Cédric Champeau wrote:
>>
>>> Hi guys,
>>>
>>> After spending some days at Devoxx, I had a few chats about Jigsaw and
>>> the implications on Groovy. I will send an email summarizing them and I
>>> think we need to come up with a plan.
>>>
>>
>> I was saying that, yes. Seems nobody wanted to believe me
>>
>> I'm not sure JIRA is the best way
>>> to track this, but in short, we have serious issues coming, and breaking
>>> changes. Maybe it's a chance for us to make all the breaking changes we
>>> thought for so long.
>>>
>>
>> I doubt it, we don't have the manpower, but we can do a bit of course
>>
>> bye Jochen
>>
>>
>>
>
>
Re: challenges through Java modules (aka jigsaw)
Posted by "Edinson E. Padrón Urdaneta" <ed...@gmail.com>.
I'm very sorry. The quote I wanted to do is:
Given the funding of groovy changed last year, this situation will show
> what kind of support exists. I know my PRs are growing long beards. Maybe
> work on this could be modularized in smaller milestones, and be pursues
> over kickstarter or GSoC.
>
I shouldn't write emails after midnight.
On Tue, Dec 8, 2015 at 3:16 AM, Edinson E. Padrón Urdaneta <
edinson.padron.urdaneta@gmail.com> wrote:
> Given the funding of groovy changed last year, this situation will show
>> what kind of support exists. I know my PRs are growing long beards.
>
>
> The Neovim project is doing something very similar [1] and it looks like
> they're doing fine.
>
> https://salt.bountysource.com/teams/neovim
>
Re: challenges through Java modules (aka jigsaw)
Posted by "Edinson E. Padrón Urdaneta" <ed...@gmail.com>.
>
> Given the funding of groovy changed last year, this situation will show
> what kind of support exists. I know my PRs are growing long beards.
The Neovim project is doing something very similar [1] and it looks like
they're doing fine.
https://salt.bountysource.com/teams/neovim
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Fri, 2015-11-27 at 20:48 +0100, Remi Forax wrote:
> Part of the problem was scons not being Python 3 compatible, part of
> the problem was a lot of too clever workarounds (using monkey
> patching) around the way file encoding was managed in the script (in
> order to try to fix the way unicode was handled in Python2, oh
> irony).
Ah, that I can see as leading to nastinesses.
SCons on Python 3 is on my agenda for Q1 2016. Too late for you I guess
in this case but people shouldn't take your experience as the state for
all time.
> so maybe someone clever will be able to fix that, but i'm not that
> guy.
I suspect you may know a person who can…
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Remi Forax <fo...@univ-mlv.fr>.
Part of the problem was scons not being Python 3 compatible, part of the problem was a lot of too clever workarounds (using monkey patching) around the way file encoding was managed in the script (in order to try to fix the way unicode was handled in Python2, oh irony).
so maybe someone clever will be able to fix that, but i'm not that guy.
Rémi
----- Mail original -----
> De: "Russel Winder" <ru...@winder.org.uk>
> À: dev@groovy.incubator.apache.org
> Envoyé: Vendredi 27 Novembre 2015 19:12:57
> Objet: Re: challenges through Java modules (aka jigsaw)
>
> On Thu, 2015-11-26 at 13:39 +0100, Remi Forax wrote:
> > Hi Cendric,
> >
> > I'm not sure it's a good idea to introduce breaking changes in
> > Groovy,
> > as an anecdote, yesterday, i've started my day by converting a build
> > script written in Python from 2 to 3, it was still not working at
> > noon, so we decide to convert it to Gradle ...
> […]
>
> My disbelief level has shot up, how can a bunch of clever people fail
> to easily convert Python 2 code to Python 3 code?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Russel Winder <ru...@winder.org.uk>.
On Thu, 2015-11-26 at 13:39 +0100, Remi Forax wrote:
> Hi Cendric,
>
> I'm not sure it's a good idea to introduce breaking changes in
> Groovy,
> as an anecdote, yesterday, i've started my day by converting a build
> script written in Python from 2 to 3, it was still not working at
> noon, so we decide to convert it to Gradle ...
[…]
My disbelief level has shot up, how can a bunch of clever people fail
to easily convert Python 2 code to Python 3 code?
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: challenges through Java modules (aka jigsaw)
Posted by Remi Forax <fo...@univ-mlv.fr>.
Hi Cendric,
I'm not sure it's a good idea to introduce breaking changes in Groovy,
as an anecdote, yesterday, i've started my day by converting a build script written in Python from 2 to 3, it was still not working at noon, so we decide to convert it to Gradle ...
You can have breaking changes in the way the classes are seen from Java (and jigsaw) without having breaking changes when you use these classes from Groovy.
By example, you can introduce a way to create type aliases to Groovy, given that, you can move Java classes around and keep the same references in the Groovy code.
Also about groovy-all, you can create a module groovy-all that will require all other modules, no ?
regards,
Rémi
----- Mail original -----
> De: "Cédric Champeau" <ce...@gmail.com>
> À: dev@groovy.incubator.apache.org
> Envoyé: Jeudi 26 Novembre 2015 13:10:11
> Objet: Re: challenges through Java modules (aka jigsaw)
> So I'm taking advantage of a bit of free time to write a quick summary of the
> situation and what we will have to decide. Basically, we have a serious
> problem (but we will not be the only ones).
> When you declare a Jigsaw module, one has to declare the list of packages
> that it exports (and sub-packages are not exported). Those packages are the
> ones that can be consumed from other Jigsaw modules. It therefore implements
> strong encapsulation, but making it impossible to reference an internal
> class of a module, both at compile time and runtime. So far so good. The
> problem is that the list of packages that belong to a module is exclusive to
> that module. So if module A exports "foo.bar", another module cannot do the
> same, or it will fail whenever the 2 modules are loaded at the same time.
> Worse, *internal packages* are also exclusive, so even if you don't export a
> package, two modules containing the same internal packages will *also* fail.
> Why is it a problem for Groovy? It is a problem because we have a very long
> history. And part of this history makes that modularization was introduced
> pretty recently, in the 2.0 release. Before, Groovy only had a single
> module. Now Groovy provides multiple modules, but also a "groovy-all" jar
> that contains all the classes from all modules. At this point, we can
> already see that this won't be possible anymore: groovy-all must die, or we
> must not support modules (sigh). But the problem is even worse. Since the
> split into modules was done late and that we wanted to preserve backwards
> compatibility, we decided not to change the packages of classes extracted
> into their own modules. So for example, XmlParser is found in groovy.util
> (not groovy.util.xml) in the groovy-xml module, but we also have lots of
> classes that live into groovy.xml into the "groovy" core module. So in
> short, our modules are no fit for Jigsaw: we must come back to the very bad
> state of having everything in a single module.
> Of course, that's not a satisfactory solution, and of course, there's nothing
> we can do to influence the decision not to make such a strong requirement:
> there are technical reasons behind this choice.
> So what can we do?
> 1. the easiest, fastest path, is to kill all modules that we have today, and
> go with a single, good old, groovy-all jar. We would go years backwards, and
> it's definitely not something we want to do. We want to have *more*
> modularization, in particular for Android, where the current split is still
> too big.
> 2. refactor modules so that each module has its own set of packages, and hope
> that we don't end up with a big groovy-all jar. Seems very unlikely.
> 3. break binary compatibility, move classes around, reorganize stuff.
> Despite my worst nightmare of fragmenting the community (that is what
> happened for all languages that tried to introduce breaking changes), I
> think we don't have the choice and we have to go route 3. And if we do so, I
> would take advantage of this fact to:
> 1. reorganize our code base even further to separate everything into their
> own modules and make them as minimal as possible
> 2. migrate to Java 8 as the minimum JDK version, which would allow us to drop
> support of the old call site caching, get a new MOP, and possibly removing
> everything that annoys us in terms of behavior which is kept for the sake of
> compatibility
> It's a long road, and it is pretty obvious that we won't be able to be ready
> for Jigsaw launch (october 2016), but it we don't decide today what we want
> to do, we will soon be obsolete, because nobody will be able to use Groovy
> with Jigsaw.
> Last but not least, as part of the Gradle team, I'd like to take advantage of
> this to rework our build to make it more consistent, faster, and aimed
> towards the future. In next January, we will start promoting a smooth
> migration path to Jigsaw using the Java software model. I think Groovy is
> still too big to migrate now to the java software model, but at least, we
> can start investigating. It is crucial to us because Gradle uses Groovy
> internally, so if Gradle doesn't run on Jigsaw, we will have a medium term
> problem (medium, because we could still run on JDK 8 but target JDK 9).
> Let the fun begin!
> 2015-11-15 17:07 GMT+01:00 Jochen Theodorou < blackdrag@gmx.org > :
> > On 15.11.2015 12:11, Cédric Champeau wrote:
>
> > > Hi guys,
> >
>
> > > After spending some days at Devoxx, I had a few chats about Jigsaw and
> >
>
> > > the implications on Groovy. I will send an email summarizing them and I
> >
>
> > > think we need to come up with a plan.
> >
>
> > I was saying that, yes. Seems nobody wanted to believe me
>
> > > I'm not sure JIRA is the best way
> >
>
> > > to track this, but in short, we have serious issues coming, and breaking
> >
>
> > > changes. Maybe it's a chance for us to make all the breaking changes we
> >
>
> > > thought for so long.
> >
>
> > I doubt it, we don't have the manpower, but we can do a bit of course
>
> > bye Jochen
>
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
So I'm taking advantage of a bit of free time to write a quick summary of
the situation and what we will have to decide. Basically, we have a serious
problem (but we will not be the only ones).
When you declare a Jigsaw module, one has to declare the list of packages
that it exports (and sub-packages are not exported). Those packages are the
ones that can be consumed from other Jigsaw modules. It therefore
implements strong encapsulation, but making it impossible to reference an
internal class of a module, both at compile time and runtime. So far so
good. The problem is that the list of packages that belong to a module is
exclusive to that module. So if module A exports "foo.bar", another module
cannot do the same, or it will fail whenever the 2 modules are loaded at
the same time. Worse, *internal packages* are also exclusive, so even if
you don't export a package, two modules containing the same internal
packages will *also* fail.
Why is it a problem for Groovy? It is a problem because we have a very long
history. And part of this history makes that modularization was introduced
pretty recently, in the 2.0 release. Before, Groovy only had a single
module. Now Groovy provides multiple modules, but also a "groovy-all" jar
that contains all the classes from all modules. At this point, we can
already see that this won't be possible anymore: groovy-all must die, or we
must not support modules (sigh). But the problem is even worse. Since the
split into modules was done late and that we wanted to preserve backwards
compatibility, we decided not to change the packages of classes extracted
into their own modules. So for example, XmlParser is found in groovy.util
(not groovy.util.xml) in the groovy-xml module, but we also have lots of
classes that live into groovy.xml into the "groovy" core module. So in
short, our modules are no fit for Jigsaw: we must come back to the very bad
state of having everything in a single module.
Of course, that's not a satisfactory solution, and of course, there's
nothing we can do to influence the decision not to make such a strong
requirement: there are technical reasons behind this choice.
So what can we do?
1. the easiest, fastest path, is to kill all modules that we have today,
and go with a single, good old, groovy-all jar. We would go years
backwards, and it's definitely not something we want to do. We want to have
*more* modularization, in particular for Android, where the current split
is still too big.
2. refactor modules so that each module has its own set of packages, and
hope that we don't end up with a big groovy-all jar. Seems very unlikely.
3. break binary compatibility, move classes around, reorganize stuff.
Despite my worst nightmare of fragmenting the community (that is what
happened for all languages that tried to introduce breaking changes), I
think we don't have the choice and we have to go route 3. And if we do so,
I would take advantage of this fact to:
1. reorganize our code base even further to separate everything into their
own modules and make them as minimal as possible
2. migrate to Java 8 as the minimum JDK version, which would allow us to
drop support of the old call site caching, get a new MOP, and possibly
removing everything that annoys us in terms of behavior which is kept for
the sake of compatibility
It's a long road, and it is pretty obvious that we won't be able to be
ready for Jigsaw launch (october 2016), but it we don't decide today what
we want to do, we will soon be obsolete, because nobody will be able to use
Groovy with Jigsaw.
Last but not least, as part of the Gradle team, I'd like to take advantage
of this to rework our build to make it more consistent, faster, and aimed
towards the future. In next January, we will start promoting a smooth
migration path to Jigsaw using the Java software model. I think Groovy is
still too big to migrate now to the java software model, but at least, we
can start investigating. It is crucial to us because Gradle uses Groovy
internally, so if Gradle doesn't run on Jigsaw, we will have a medium term
problem (medium, because we could still run on JDK 8 but target JDK 9).
Let the fun begin!
2015-11-15 17:07 GMT+01:00 Jochen Theodorou <bl...@gmx.org>:
> On 15.11.2015 12:11, Cédric Champeau wrote:
>
>> Hi guys,
>>
>> After spending some days at Devoxx, I had a few chats about Jigsaw and
>> the implications on Groovy. I will send an email summarizing them and I
>> think we need to come up with a plan.
>>
>
> I was saying that, yes. Seems nobody wanted to believe me
>
> I'm not sure JIRA is the best way
>> to track this, but in short, we have serious issues coming, and breaking
>> changes. Maybe it's a chance for us to make all the breaking changes we
>> thought for so long.
>>
>
> I doubt it, we don't have the manpower, but we can do a bit of course
>
> bye Jochen
>
>
>
Re: challenges through Java modules (aka jigsaw)
Posted by Jochen Theodorou <bl...@gmx.org>.
On 15.11.2015 12:11, Cédric Champeau wrote:
> Hi guys,
>
> After spending some days at Devoxx, I had a few chats about Jigsaw and
> the implications on Groovy. I will send an email summarizing them and I
> think we need to come up with a plan.
I was saying that, yes. Seems nobody wanted to believe me
> I'm not sure JIRA is the best way
> to track this, but in short, we have serious issues coming, and breaking
> changes. Maybe it's a chance for us to make all the breaking changes we
> thought for so long.
I doubt it, we don't have the manpower, but we can do a bit of course
bye Jochen
Re: challenges through Java modules (aka jigsaw)
Posted by Cédric Champeau <ce...@gmail.com>.
Hi guys,
After spending some days at Devoxx, I had a few chats about Jigsaw and the
implications on Groovy. I will send an email summarizing them and I think
we need to come up with a plan. I'm not sure JIRA is the best way to track
this, but in short, we have serious issues coming, and breaking changes.
Maybe it's a chance for us to make all the breaking changes we thought for
so long.
2015-10-09 22:48 GMT+02:00 Thibault Kruse <ti...@googlemail.com>:
> So you mention 3 potential problem areas, reflection on flow types,
> Grapes and the URLClassloader, and module descriptors.
>
> Would it be more resonable to open JIRA issues on those to collect
> ideas and resources?
>
> Regarding mthe URLClassloader, all I found was this:
>
> https://discuss.gradle.org/t/gradle-is-broken-by-jdk9-application-class-loader/9206
> http://comments.gmane.org/gmane.comp.java.openjdk.jigsaw/969
>
> On Thu, Oct 8, 2015 at 1:08 PM, Jochen Theodorou <bl...@gmx.org>
> wrote:
> > Hi all,
> >
> >
> > With Jigsaw here will be some changes affecting Groovy in some major
> ways,
> > but frankly it is still unclear to me as of how much. This is partially
> to
> > Jigsaw still being so young. This mail is mostly to write down some
> thoughts
> > for sharing.
> >
> > Jigsaw means that we will not have access to all classes we have now.
> Many
> > factory methods of the JDK for example will provide such classes. Jigsaw
> > means we can still get a class object from it, but we cannot get the
> methods
> > or do invocations directly with the implementation class. Things need to
> get
> > through the public base class.
> >
> > This is for example a problem with flow typing in static compilation:
> >
> > class Base { def foo(){}; static baseImpl(){return new Hidden()}}
> >
> > class Hidden extends Base {}
> >
> >
> >
> > Hidden h = Base.baseImpl() //works
> >
> > h.foo() // works not
> >
> > Base b = h
> >
> > b.foo() // works (in Java at least)
> >
> > h.foo() fails in Java because Hidden is used as base class and protected
> by
> > the module system. Doing the same invocation using the public available
> > class will work though. This of course will here still invoke Hidden#foo
> > thanks to it being a virtual method call.
> >
> > For static compilation the problem is, that the trick with Base b might
> not
> > work. b will still have the flow type Hidden. We need to ensure we do the
> > invocation not with the flow type, if the method is also available in the
> > normal static type.
> >
> > But now that I think of it, the compiler might already have a problem,
> since
> > the compiler can probably not inspect Hidden by Reflection at all. This
> > means we need to check the code generating ClassNodes how it reacts to
> > exceptions being thrown due to the class object being available, but not
> the
> > members.
> >
> > This also means, that if we look for a class sing reflection like Unsafe,
> > and assume, just because the class is there, that we can use it, we have
> a
> > code part to change. I am thinking of Unsafe hereof course, but also
> about
> > that special interface we use for callsite caching, that bypasses
> > verification. There is probably more.
> >
> > And of course the Groovy runtime has similar problems. We have already
> some
> > checks for classes not being available through strict SecurityManagers,
> but
> > new exceptions (JDK9 specific ones) can be a problem.. and most probably
> > several places are involved
> >
> > Next problem is with @Grab. It looks like there will be no longer a
> > URLClassLoader from the JDK available. Thus we can no longer add a jar at
> > runtime. RootLoader (used when executing Groovy from the console) can
> still
> > be used of course, but does not cover the system loader case, which
> databses
> > drivers may need. It is yet unclear to me how the module system works at
> > runtime with database drivers added at runtime as well.
> >
> > Then I see a conceptual problem for Groovy as well. If a module is
> supposed
> > to be written in Groovy and has a private part, then the Groovy runtime
> > cannot really access those classes and for example cannot call private
> > methods in those classes. To give the runtime access I think Groovy
> itself
> > would need a module descriptor. But unless I misunderstand the proposal,
> > this means a random script can access those private classes from that
> module
> > written in Groovy as well. Why? Because, if it does a method call through
> > the runtime, and if the runtime has access, then the script has access.
> >
> > There is a concept called Layers, which I have not yet found enough
> > information about, which may help here... not sure yet.
> >
> > bye blackdrag
> >
> > --
> > Jochen "blackdrag" Theodorou
> > blog: http://blackdragsview.blogspot.com/
> >
>
Re: challenges through Java modules (aka jigsaw)
Posted by Thibault Kruse <ti...@googlemail.com>.
So you mention 3 potential problem areas, reflection on flow types,
Grapes and the URLClassloader, and module descriptors.
Would it be more resonable to open JIRA issues on those to collect
ideas and resources?
Regarding mthe URLClassloader, all I found was this:
https://discuss.gradle.org/t/gradle-is-broken-by-jdk9-application-class-loader/9206
http://comments.gmane.org/gmane.comp.java.openjdk.jigsaw/969
On Thu, Oct 8, 2015 at 1:08 PM, Jochen Theodorou <bl...@gmx.org> wrote:
> Hi all,
>
>
> With Jigsaw here will be some changes affecting Groovy in some major ways,
> but frankly it is still unclear to me as of how much. This is partially to
> Jigsaw still being so young. This mail is mostly to write down some thoughts
> for sharing.
>
> Jigsaw means that we will not have access to all classes we have now. Many
> factory methods of the JDK for example will provide such classes. Jigsaw
> means we can still get a class object from it, but we cannot get the methods
> or do invocations directly with the implementation class. Things need to get
> through the public base class.
>
> This is for example a problem with flow typing in static compilation:
>
> class Base { def foo(){}; static baseImpl(){return new Hidden()}}
>
> class Hidden extends Base {}
>
>
>
> Hidden h = Base.baseImpl() //works
>
> h.foo() // works not
>
> Base b = h
>
> b.foo() // works (in Java at least)
>
> h.foo() fails in Java because Hidden is used as base class and protected by
> the module system. Doing the same invocation using the public available
> class will work though. This of course will here still invoke Hidden#foo
> thanks to it being a virtual method call.
>
> For static compilation the problem is, that the trick with Base b might not
> work. b will still have the flow type Hidden. We need to ensure we do the
> invocation not with the flow type, if the method is also available in the
> normal static type.
>
> But now that I think of it, the compiler might already have a problem, since
> the compiler can probably not inspect Hidden by Reflection at all. This
> means we need to check the code generating ClassNodes how it reacts to
> exceptions being thrown due to the class object being available, but not the
> members.
>
> This also means, that if we look for a class sing reflection like Unsafe,
> and assume, just because the class is there, that we can use it, we have a
> code part to change. I am thinking of Unsafe hereof course, but also about
> that special interface we use for callsite caching, that bypasses
> verification. There is probably more.
>
> And of course the Groovy runtime has similar problems. We have already some
> checks for classes not being available through strict SecurityManagers, but
> new exceptions (JDK9 specific ones) can be a problem.. and most probably
> several places are involved
>
> Next problem is with @Grab. It looks like there will be no longer a
> URLClassLoader from the JDK available. Thus we can no longer add a jar at
> runtime. RootLoader (used when executing Groovy from the console) can still
> be used of course, but does not cover the system loader case, which databses
> drivers may need. It is yet unclear to me how the module system works at
> runtime with database drivers added at runtime as well.
>
> Then I see a conceptual problem for Groovy as well. If a module is supposed
> to be written in Groovy and has a private part, then the Groovy runtime
> cannot really access those classes and for example cannot call private
> methods in those classes. To give the runtime access I think Groovy itself
> would need a module descriptor. But unless I misunderstand the proposal,
> this means a random script can access those private classes from that module
> written in Groovy as well. Why? Because, if it does a method call through
> the runtime, and if the runtime has access, then the script has access.
>
> There is a concept called Layers, which I have not yet found enough
> information about, which may help here... not sure yet.
>
> bye blackdrag
>
> --
> Jochen "blackdrag" Theodorou
> blog: http://blackdragsview.blogspot.com/
>