You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Uwe Schindler <us...@apache.org> on 2016/12/16 09:58:35 UTC
RE: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch
Hi Jochen,
thank you for the information! Is there any plan about a release? I also found no JIRA issue about this issue to link it against our JIRA: https://issues.apache.org/jira/browse/LUCENE-7596
The problem makes our build system unusable, so it would be very important to have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it would be good to have the bugfix release available there, which requires a release. I think the same applies for Gradle users (Elasticsearch).
As a temporary workaround we might be able to use the Apache Snapshot repository, but this is not allowed if we do a release of Lucene.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> -----Original Message-----
> From: Jochen Theodorou [mailto:blackdrag@gmx.org]
> Sent: Saturday, December 10, 2016 9:23 AM
> To: Uwe Schindler <us...@apache.org>; jigsaw-dev@openjdk.java.net;
> Core-Libs-Dev <co...@openjdk.java.net>
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
>
> On 09.12.2016 23:32, Uwe Schindler wrote:
> > Hi,
> >
> > I updated our Jenkins server for the JDK 9 preview testing to use build 148.
> Previously we had build 140 and build 147, which both worked without any
> issues. But after the update the following stuff goes wrong:
> >
> > (1) Unmapping of direct buffers no longer works, although this API was
> marked as critical because there is no replacement up to now, so code can
> unmap memory mapped files, which is one of the most important things
> Apache Lucene needs to use to access huge random access files while
> reading the index. Without memory mapping, the slowdown for Lucene
> users will be huge
> >
> > This is caused by the recent Jigsaw changes, published in build 148.
> Unfortunately we did not test the Jigsaw builds, so we would have noticed
> that earlier. Basically the following peace of code fails now (with or without
> doPrivileged and with/without security manager):
> >
> > final Class<?> directBufferClass =
> Class.forName("java.nio.DirectByteBuffer");
> >
> > final Method m = directBufferClass.getMethod("cleaner");
> > m.setAccessible(true);
> > MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
> > Class<?> cleanerClass =
> directBufferCleanerMethod.type().returnType();
> > // build method handle for unmapping, full code is here:
> https://goo.gl/TfQWl6
>
> I guess that is the effect of #AwkwardStrongEncapsulation. I would
> advise doing regular checks against the jigsaw builds to know about such
> problems in the future earlier... but seeing your code break without an
> obvious good solution sure is stressful. I feel with you.
>
> [...]
> > (2) A second thing we noticed is that Groovy no longer works and dies with
> strange error messages.
>
> That is because versions including Groovy 2.4.7 are using
> setAccessible(AccessibleObject[] array, true), and the array will also
> include private methods or fields. This worked till
> #AwkwardStrongEncapsulation because will then a class was either
> exported and its method can all be made accessible or not. For example
> on GAE or earlier versions of the module system. Now an exported class
> may break this, since its private methods can no longer be made
> accessible using setAccessible.
>
> A fix for this is already committed, we are only waiting for release of
> Groovy 2.4.8. Of course even with the fix Groovy code can possibly
> break... for example if you did the direct buffer access in Groovy.
>
> Btw, do not hesitate to ask about such problems on groovy-user, please.
>
> bye Jochen
RE: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch
Posted by Uwe Schindler <us...@apache.org>.
Hi,
sorry for the delay! I updated Lucene's build system to use the JFrog snapshort artifacts and build succeeds with JDK 9 b148+:
That's the one that was choosen by Apache Ant's Ivy downloader: groovy-all-2.4.8-20161220.101835-40.jar
So we are waiting for a release!
Uwe
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: core-libs-dev [mailto:core-libs-dev-bounces@openjdk.java.net] On
> Behalf Of Jochen Theodorou
> Sent: Friday, December 16, 2016 2:11 PM
> To: dev@groovy.apache.org; jigsaw-dev@openjdk.java.net; 'Core-Libs-Dev'
> <co...@openjdk.java.net>
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
>
> Hi,
>
> I strongly hope Paul and Cedric will be able to start the release
> process next week, if not we will have to do it the old way I think.
>
> what would help us a lot would be you testing the GROOVY_2_4_X branch
> with your build system to see if it really does solve your problem. Even
> if it is only locally on your computer
>
> bye Jochen
>
> On 16.12.2016 10:58, Uwe Schindler wrote:
> > Hi Jochen,
> >
> > thank you for the information! Is there any plan about a release? I also
> found no JIRA issue about this issue to link it against our JIRA:
> https://issues.apache.org/jira/browse/LUCENE-7596
> >
> > The problem makes our build system unusable, so it would be very
> important to have a fix quite soon! As our Ant/Ivy-based build relies on
> Maven Central, it would be good to have the bugfix release available there,
> which requires a release. I think the same applies for Gradle users
> (Elasticsearch).
> >
> > As a temporary workaround we might be able to use the Apache Snapshot
> repository, but this is not allowed if we do a release of Lucene.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> >> -----Original Message-----
> >> From: Jochen Theodorou [mailto:blackdrag@gmx.org]
> >> Sent: Saturday, December 10, 2016 9:23 AM
> >> To: Uwe Schindler <us...@apache.org>; jigsaw-
> dev@openjdk.java.net;
> >> Core-Libs-Dev <co...@openjdk.java.net>
> >> Subject: Re: Java 9 build 148 causes trouble in Apache
> >> Lucene/Solr/Elasticsearch
> >>
> >> On 09.12.2016 23:32, Uwe Schindler wrote:
> >>> Hi,
> >>>
> >>> I updated our Jenkins server for the JDK 9 preview testing to use build
> 148.
> >> Previously we had build 140 and build 147, which both worked without
> any
> >> issues. But after the update the following stuff goes wrong:
> >>>
> >>> (1) Unmapping of direct buffers no longer works, although this API was
> >> marked as critical because there is no replacement up to now, so code can
> >> unmap memory mapped files, which is one of the most important things
> >> Apache Lucene needs to use to access huge random access files while
> >> reading the index. Without memory mapping, the slowdown for Lucene
> >> users will be huge
> >>>
> >>> This is caused by the recent Jigsaw changes, published in build 148.
> >> Unfortunately we did not test the Jigsaw builds, so we would have noticed
> >> that earlier. Basically the following peace of code fails now (with or
> without
> >> doPrivileged and with/without security manager):
> >>>
> >>> final Class<?> directBufferClass =
> >> Class.forName("java.nio.DirectByteBuffer");
> >>>
> >>> final Method m = directBufferClass.getMethod("cleaner");
> >>> m.setAccessible(true);
> >>> MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
> >>> Class<?> cleanerClass =
> >> directBufferCleanerMethod.type().returnType();
> >>> // build method handle for unmapping, full code is here:
> >> https://goo.gl/TfQWl6
> >>
> >> I guess that is the effect of #AwkwardStrongEncapsulation. I would
> >> advise doing regular checks against the jigsaw builds to know about such
> >> problems in the future earlier... but seeing your code break without an
> >> obvious good solution sure is stressful. I feel with you.
> >>
> >> [...]
> >>> (2) A second thing we noticed is that Groovy no longer works and dies
> with
> >> strange error messages.
> >>
> >> That is because versions including Groovy 2.4.7 are using
> >> setAccessible(AccessibleObject[] array, true), and the array will also
> >> include private methods or fields. This worked till
> >> #AwkwardStrongEncapsulation because will then a class was either
> >> exported and its method can all be made accessible or not. For example
> >> on GAE or earlier versions of the module system. Now an exported class
> >> may break this, since its private methods can no longer be made
> >> accessible using setAccessible.
> >>
> >> A fix for this is already committed, we are only waiting for release of
> >> Groovy 2.4.8. Of course even with the fix Groovy code can possibly
> >> break... for example if you did the direct buffer access in Groovy.
> >>
> >> Btw, do not hesitate to ask about such problems on groovy-user, please.
> >>
> >> bye Jochen
> >
Re: Java 9 build 148 causes trouble in Apache
Lucene/Solr/Elasticsearch
Posted by Jochen Theodorou <bl...@gmx.org>.
Hi,
I strongly hope Paul and Cedric will be able to start the release
process next week, if not we will have to do it the old way I think.
what would help us a lot would be you testing the GROOVY_2_4_X branch
with your build system to see if it really does solve your problem. Even
if it is only locally on your computer
bye Jochen
On 16.12.2016 10:58, Uwe Schindler wrote:
> Hi Jochen,
>
> thank you for the information! Is there any plan about a release? I also found no JIRA issue about this issue to link it against our JIRA: https://issues.apache.org/jira/browse/LUCENE-7596
>
> The problem makes our build system unusable, so it would be very important to have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it would be good to have the bugfix release available there, which requires a release. I think the same applies for Gradle users (Elasticsearch).
>
> As a temporary workaround we might be able to use the Apache Snapshot repository, but this is not allowed if we do a release of Lucene.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>> -----Original Message-----
>> From: Jochen Theodorou [mailto:blackdrag@gmx.org]
>> Sent: Saturday, December 10, 2016 9:23 AM
>> To: Uwe Schindler <us...@apache.org>; jigsaw-dev@openjdk.java.net;
>> Core-Libs-Dev <co...@openjdk.java.net>
>> Subject: Re: Java 9 build 148 causes trouble in Apache
>> Lucene/Solr/Elasticsearch
>>
>> On 09.12.2016 23:32, Uwe Schindler wrote:
>>> Hi,
>>>
>>> I updated our Jenkins server for the JDK 9 preview testing to use build 148.
>> Previously we had build 140 and build 147, which both worked without any
>> issues. But after the update the following stuff goes wrong:
>>>
>>> (1) Unmapping of direct buffers no longer works, although this API was
>> marked as critical because there is no replacement up to now, so code can
>> unmap memory mapped files, which is one of the most important things
>> Apache Lucene needs to use to access huge random access files while
>> reading the index. Without memory mapping, the slowdown for Lucene
>> users will be huge
>>>
>>> This is caused by the recent Jigsaw changes, published in build 148.
>> Unfortunately we did not test the Jigsaw builds, so we would have noticed
>> that earlier. Basically the following peace of code fails now (with or without
>> doPrivileged and with/without security manager):
>>>
>>> final Class<?> directBufferClass =
>> Class.forName("java.nio.DirectByteBuffer");
>>>
>>> final Method m = directBufferClass.getMethod("cleaner");
>>> m.setAccessible(true);
>>> MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
>>> Class<?> cleanerClass =
>> directBufferCleanerMethod.type().returnType();
>>> // build method handle for unmapping, full code is here:
>> https://goo.gl/TfQWl6
>>
>> I guess that is the effect of #AwkwardStrongEncapsulation. I would
>> advise doing regular checks against the jigsaw builds to know about such
>> problems in the future earlier... but seeing your code break without an
>> obvious good solution sure is stressful. I feel with you.
>>
>> [...]
>>> (2) A second thing we noticed is that Groovy no longer works and dies with
>> strange error messages.
>>
>> That is because versions including Groovy 2.4.7 are using
>> setAccessible(AccessibleObject[] array, true), and the array will also
>> include private methods or fields. This worked till
>> #AwkwardStrongEncapsulation because will then a class was either
>> exported and its method can all be made accessible or not. For example
>> on GAE or earlier versions of the module system. Now an exported class
>> may break this, since its private methods can no longer be made
>> accessible using setAccessible.
>>
>> A fix for this is already committed, we are only waiting for release of
>> Groovy 2.4.8. Of course even with the fix Groovy code can possibly
>> break... for example if you did the direct buffer access in Groovy.
>>
>> Btw, do not hesitate to ask about such problems on groovy-user, please.
>>
>> bye Jochen
>