You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jeff Courtade <co...@gmail.com> on 2018/09/26 14:00:45 UTC
Java version 11 for solr 7.5?
Hello,
We are looking to migrate to solr 7.5 with java 11 from solr 4.3.0 with
java 7.
I have a couple basic questions...
What version of Java is current solr 7.5 development and testing based on?
Can we use java 11 with solr 7.5? any known issues?
Can we use GC1 garbage collection yet or do we still need to use CMS?
--
Thanks,
Jeff Courtade
M: 240.507.6116
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
Can you tell me where I could get insight into the testing cycles and
results?
On Wed, Sep 26, 2018, 1:03 PM Erick Erickson <er...@gmail.com>
wrote:
> There are consistent failures under JDK 11 in the automated tests that
> Solr/Lucene runs that do not happen for other releases. I personally
> haven't tried diving into them to know whether they're test artifacts
> or not.
>
> JDK 9 and JKD 10 also have open issues, especially around Hadoop
> integration.
>
> I would recommend sticking with JDK 8 for the time being, that's the
> most tested/used version of Solr. Track
> https://issues.apache.org/jira/browse/SOLR-12809 for what progress is
> made with more recent Java versions.
>
> Best,
> Erick
> On Wed, Sep 26, 2018 at 8:44 AM Markus Jelsma
> <ma...@openindex.io> wrote:
> >
> > Indeed, but JDK-8038348 has been fixed very recently for Java 9 or
> higher.
> >
> > -----Original message-----
> > > From:Jeff Courtade <co...@gmail.com>
> > > Sent: Wednesday 26th September 2018 17:36
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Java version 11 for solr 7.5?
> > >
> > > My concern with using g1 is solely based on finding this.
> > > Does anyone have any information on this?
> > >
> > >
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
> > >
> > > "Do not, under any circumstances, run Lucene with the G1 garbage
> collector.
> > > Lucene's test suite fails with the G1 garbage collector on a regular
> basis,
> > > including bugs that cause index corruption. There is no person on this
> > > planet that seems to understand such bugs (see
> > > https://bugs.openjdk.java.net/browse/JDK-8038348, open for over a
> year), so
> > > don't count on the situation changing soon. This information is not
> out of
> > > date, and don't think that the next oracle java release will fix the
> > > situation."
> > >
> > >
> > > On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood <
> wunder@wunderwood.org>
> > > wrote:
> > >
> > > > We’ve been running G1 in prod for at least 18 months. Our biggest
> cluster
> > > > is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on
> our
> > > > 4.10.4 master/slave cluster.
> > > >
> > > > wunder
> > > > Walter Underwood
> > > > wunder@wunderwood.org
> > > > http://observer.wunderwood.org/ (my blog)
> > > >
> > > > > On Sep 26, 2018, at 7:37 AM, Jeff Courtade <courtadejeff@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > Thanks for that...
> > > > > I am just starting to look at this I was unaware of the license
> debacle.
> > > > >
> > > > > Automated testing up to 10 is great.
> > > > >
> > > > > I am still curious about the GC1 being supported now...
> > > > >
> > > > > On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com>
> wrote:
> > > > >
> > > > >> Jeff Courtade wrote
> > > > >>> Can we use GC1 garbage collection yet or do we still need to use
> CMS?
> > > > >>
> > > > >> I believe you should be safe to go with G1. We've applied it in
> in a
> > > > Solr
> > > > >> 6.6 cluster with 10 shards, 3 replicas per shard and an index of
> about
> > > > >> 500GB
> > > > >> (1,5T counting all replicas) and it works extremely well
> (throughput >
> > > > >> 99%).
> > > > >> The use-case includes complex search queries and faceting.
> > > > >> There is also this post you can use as a starting point
> > > > >>
> > > > >>
> > > >
> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Sent from:
> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > > > >>
> > > > > --
> > > > >
> > > > > Jeff Courtade
> > > > > M: 240.507.6116 <(240)%20507-6116>
> > > >
> > > > --
> > >
> > > Jeff Courtade
> > > M: 240.507.6116
> > >
>
Re: Java version 11 for solr 7.5?
Posted by Erick Erickson <er...@gmail.com>.
There are consistent failures under JDK 11 in the automated tests that
Solr/Lucene runs that do not happen for other releases. I personally
haven't tried diving into them to know whether they're test artifacts
or not.
JDK 9 and JKD 10 also have open issues, especially around Hadoop integration.
I would recommend sticking with JDK 8 for the time being, that's the
most tested/used version of Solr. Track
https://issues.apache.org/jira/browse/SOLR-12809 for what progress is
made with more recent Java versions.
Best,
Erick
On Wed, Sep 26, 2018 at 8:44 AM Markus Jelsma
<ma...@openindex.io> wrote:
>
> Indeed, but JDK-8038348 has been fixed very recently for Java 9 or higher.
>
> -----Original message-----
> > From:Jeff Courtade <co...@gmail.com>
> > Sent: Wednesday 26th September 2018 17:36
> > To: solr-user@lucene.apache.org
> > Subject: Re: Java version 11 for solr 7.5?
> >
> > My concern with using g1 is solely based on finding this.
> > Does anyone have any information on this?
> >
> > https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
> >
> > "Do not, under any circumstances, run Lucene with the G1 garbage collector.
> > Lucene's test suite fails with the G1 garbage collector on a regular basis,
> > including bugs that cause index corruption. There is no person on this
> > planet that seems to understand such bugs (see
> > https://bugs.openjdk.java.net/browse/JDK-8038348, open for over a year), so
> > don't count on the situation changing soon. This information is not out of
> > date, and don't think that the next oracle java release will fix the
> > situation."
> >
> >
> > On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood <wu...@wunderwood.org>
> > wrote:
> >
> > > We’ve been running G1 in prod for at least 18 months. Our biggest cluster
> > > is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on our
> > > 4.10.4 master/slave cluster.
> > >
> > > wunder
> > > Walter Underwood
> > > wunder@wunderwood.org
> > > http://observer.wunderwood.org/ (my blog)
> > >
> > > > On Sep 26, 2018, at 7:37 AM, Jeff Courtade <co...@gmail.com>
> > > wrote:
> > > >
> > > > Thanks for that...
> > > > I am just starting to look at this I was unaware of the license debacle.
> > > >
> > > > Automated testing up to 10 is great.
> > > >
> > > > I am still curious about the GC1 being supported now...
> > > >
> > > > On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com> wrote:
> > > >
> > > >> Jeff Courtade wrote
> > > >>> Can we use GC1 garbage collection yet or do we still need to use CMS?
> > > >>
> > > >> I believe you should be safe to go with G1. We've applied it in in a
> > > Solr
> > > >> 6.6 cluster with 10 shards, 3 replicas per shard and an index of about
> > > >> 500GB
> > > >> (1,5T counting all replicas) and it works extremely well (throughput >
> > > >> 99%).
> > > >> The use-case includes complex search queries and faceting.
> > > >> There is also this post you can use as a starting point
> > > >>
> > > >>
> > > http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > > >>
> > > > --
> > > >
> > > > Jeff Courtade
> > > > M: 240.507.6116 <(240)%20507-6116>
> > >
> > > --
> >
> > Jeff Courtade
> > M: 240.507.6116
> >
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
Thanks ..!
On Wed, Sep 26, 2018 at 11:44 AM Markus Jelsma <ma...@openindex.io>
wrote:
> Indeed, but JDK-8038348 has been fixed very recently for Java 9 or higher.
>
> -----Original message-----
> > From:Jeff Courtade <co...@gmail.com>
> > Sent: Wednesday 26th September 2018 17:36
> > To: solr-user@lucene.apache.org
> > Subject: Re: Java version 11 for solr 7.5?
> >
> > My concern with using g1 is solely based on finding this.
> > Does anyone have any information on this?
> >
> >
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
> >
> > "Do not, under any circumstances, run Lucene with the G1 garbage
> collector.
> > Lucene's test suite fails with the G1 garbage collector on a regular
> basis,
> > including bugs that cause index corruption. There is no person on this
> > planet that seems to understand such bugs (see
> > https://bugs.openjdk.java.net/browse/JDK-8038348, open for over a
> year), so
> > don't count on the situation changing soon. This information is not out
> of
> > date, and don't think that the next oracle java release will fix the
> > situation."
> >
> >
> > On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood <wunder@wunderwood.org
> >
> > wrote:
> >
> > > We’ve been running G1 in prod for at least 18 months. Our biggest
> cluster
> > > is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on our
> > > 4.10.4 master/slave cluster.
> > >
> > > wunder
> > > Walter Underwood
> > > wunder@wunderwood.org
> > > http://observer.wunderwood.org/ (my blog)
> > >
> > > > On Sep 26, 2018, at 7:37 AM, Jeff Courtade <co...@gmail.com>
> > > wrote:
> > > >
> > > > Thanks for that...
> > > > I am just starting to look at this I was unaware of the license
> debacle.
> > > >
> > > > Automated testing up to 10 is great.
> > > >
> > > > I am still curious about the GC1 being supported now...
> > > >
> > > > On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com>
> wrote:
> > > >
> > > >> Jeff Courtade wrote
> > > >>> Can we use GC1 garbage collection yet or do we still need to use
> CMS?
> > > >>
> > > >> I believe you should be safe to go with G1. We've applied it in in a
> > > Solr
> > > >> 6.6 cluster with 10 shards, 3 replicas per shard and an index of
> about
> > > >> 500GB
> > > >> (1,5T counting all replicas) and it works extremely well
> (throughput >
> > > >> 99%).
> > > >> The use-case includes complex search queries and faceting.
> > > >> There is also this post you can use as a starting point
> > > >>
> > > >>
> > >
> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Sent from:
> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > > >>
> > > > --
> > > >
> > > > Jeff Courtade
> > > > M: 240.507.6116 <(240)%20507-6116> <(240)%20507-6116>
> > >
> > > --
> >
> > Jeff Courtade
> > M: 240.507.6116 <(240)%20507-6116>
> >
>
--
Jeff Courtade
M: 240.507.6116
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
This is true.
I am thinking isf solr says 8 and up it really is 8 and up there is no
other reference I find to not using G1 collection.
The java support right now for old versions is really a mess. Currently if
you want ongoing support patches without an Oracle support contract the
only way to achieve that in October 2018 is Oracle 11.
If you are doing production or commercial work you have to use openjdk or
buy a license. Such a mess
On Wed, Sep 26, 2018, 4:04 PM Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Jeff,
>
> On 9/26/18 11:35, Jeff Courtade wrote:
> > My concern with using g1 is solely based on finding this. Does
> > anyone have any information on this?
> >
> > https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_
> .2F_OpenJDK_Bugs
> >
> > "Do not, under any circumstances, run Lucene with the G1 garbage
> > collector. Lucene's test suite fails with the G1 garbage collector
> > on a regular basis, including bugs that cause index corruption.
> > There is no person on this planet that seems to understand such
> > bugs (see https://bugs.openjdk.java.net/browse/JDK-8038348, open
> > for over a year), so don't count on the situation changing soon.
> > This information is not out of date, and don't think that the next
> > oracle java release will fix the situation."
>
> That language is 3 years old and likely just hasn't been updated after
> it was no longer relevant. Also, it isn't attributed to anyone in
> particular (it's anonymous), so ... maybe it was one person's opinion
> and not a project-initiated warning.
>
> - -chris
>
> > On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood
> > <wu...@wunderwood.org> wrote:
> >
> >> We’ve been running G1 in prod for at least 18 months. Our biggest
> >> cluster is 48 machines, each with 36 CPUs, running 6.6.2. We also
> >> run it on our 4.10.4 master/slave cluster.
> >>
> >> wunder Walter Underwood wunder@wunderwood.org
> >> http://observer.wunderwood.org/ (my blog)
> >>
> >>> On Sep 26, 2018, at 7:37 AM, Jeff Courtade
> >>> <co...@gmail.com>
> >> wrote:
> >>>
> >>> Thanks for that... I am just starting to look at this I was
> >>> unaware of the license debacle.
> >>>
> >>> Automated testing up to 10 is great.
> >>>
> >>> I am still curious about the GC1 being supported now...
> >>>
> >>> On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com>
> >>> wrote:
> >>>
> >>>> Jeff Courtade wrote
> >>>>> Can we use GC1 garbage collection yet or do we still need
> >>>>> to use CMS?
> >>>>
> >>>> I believe you should be safe to go with G1. We've applied it
> >>>> in in a
> >> Solr
> >>>> 6.6 cluster with 10 shards, 3 replicas per shard and an index
> >>>> of about 500GB (1,5T counting all replicas) and it works
> >>>> extremely well (throughput > 99%). The use-case includes
> >>>> complex search queries and faceting. There is also this post
> >>>> you can use as a starting point
> >>>>
> >>>>
> >> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-p
> roduction/
> <http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>
> - --
> >>>> Sent from:
> >>>> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >>>>
> >>> --
> >>>
> >>> Jeff Courtade M: 240.507.6116 <(240)%20507-6116>
> >>
> >> --
> >
> > Jeff Courtade M: 240.507.6116
> >
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlur5i0ACgkQHPApP6U8
> pFioLw/9HzPmNo1wtsfZDIsJjXy4i+F0YYqBFKRqXcPH8mLgpxicE5enRVI9he6p
> 1Z3Mz0wzFj/H91eWktmGyNSKSmFjkYI2IgCBrsZv1gPDFn3mI3TwapJgTR0J4GAg
> wXB/9GRHuCCTz7qvfQexBOwOt25OKVOhcvNFVI8bxV0hFl58Nlo56Qzt33X/JS32
> jH2jIlz77pal1t5ZhnXJwCSWQyWsLnr5GtoxDisvvOl1o3Ey/WIllvCe8x7M+PvA
> 0/DIK/5niTSCwcv0LVCPIWsE/HCjsSWfdhnhtTnu1088OTKwb2dsa7wyBJItZUzw
> fCTcmcGclViGUa2QAnXNFiVPj1y0PhFxAPMCU6mWPerCSH6cYn5neicsp2AYovoj
> dRcs4LGrGf0S7PVJBq/DQdb44XbzvFkkp2SjS9WAnLpBv7RwP4bWfDvMCJsZWJOU
> 8J2r4ZbkVUjByQ3mAXMZN7bKC6hHBQLLzAwodloAV0OWHJ+Io96flTclDRPt4N6e
> J8olEQezDKcgkZDg0GV8I9WxUzeTHI+QvnZxUzwsT/sJUPgxjSDjHlous5HU29ay
> 6lynoEjVFJd4yYAwh6gaRPMw34xKFT6a62D6bDmcL0MqPCpbcbOny+kgx0k7bzl5
> FNsapJ5vCIaG0/tPTuWEY/jaqmhNNznXDr+sEX5l8Sk1ZQz8+/U=
> =Y8qg
> -----END PGP SIGNATURE-----
>
Re: Java version 11 for solr 7.5?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Jeff,
On 9/26/18 11:35, Jeff Courtade wrote:
> My concern with using g1 is solely based on finding this. Does
> anyone have any information on this?
>
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_
.2F_OpenJDK_Bugs
>
> "Do not, under any circumstances, run Lucene with the G1 garbage
> collector. Lucene's test suite fails with the G1 garbage collector
> on a regular basis, including bugs that cause index corruption.
> There is no person on this planet that seems to understand such
> bugs (see https://bugs.openjdk.java.net/browse/JDK-8038348, open
> for over a year), so don't count on the situation changing soon.
> This information is not out of date, and don't think that the next
> oracle java release will fix the situation."
That language is 3 years old and likely just hasn't been updated after
it was no longer relevant. Also, it isn't attributed to anyone in
particular (it's anonymous), so ... maybe it was one person's opinion
and not a project-initiated warning.
- -chris
> On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood
> <wu...@wunderwood.org> wrote:
>
>> We’ve been running G1 in prod for at least 18 months. Our biggest
>> cluster is 48 machines, each with 36 CPUs, running 6.6.2. We also
>> run it on our 4.10.4 master/slave cluster.
>>
>> wunder Walter Underwood wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>>> On Sep 26, 2018, at 7:37 AM, Jeff Courtade
>>> <co...@gmail.com>
>> wrote:
>>>
>>> Thanks for that... I am just starting to look at this I was
>>> unaware of the license debacle.
>>>
>>> Automated testing up to 10 is great.
>>>
>>> I am still curious about the GC1 being supported now...
>>>
>>> On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com>
>>> wrote:
>>>
>>>> Jeff Courtade wrote
>>>>> Can we use GC1 garbage collection yet or do we still need
>>>>> to use CMS?
>>>>
>>>> I believe you should be safe to go with G1. We've applied it
>>>> in in a
>> Solr
>>>> 6.6 cluster with 10 shards, 3 replicas per shard and an index
>>>> of about 500GB (1,5T counting all replicas) and it works
>>>> extremely well (throughput > 99%). The use-case includes
>>>> complex search queries and faceting. There is also this post
>>>> you can use as a starting point
>>>>
>>>>
>> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-p
roduction/
>>>>
>>>>
>>>>
>>>>
>>>>
>>
- --
>>>> Sent from:
>>>> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>>>
>>> --
>>>
>>> Jeff Courtade M: 240.507.6116 <(240)%20507-6116>
>>
>> --
>
> Jeff Courtade M: 240.507.6116
>
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlur5i0ACgkQHPApP6U8
pFioLw/9HzPmNo1wtsfZDIsJjXy4i+F0YYqBFKRqXcPH8mLgpxicE5enRVI9he6p
1Z3Mz0wzFj/H91eWktmGyNSKSmFjkYI2IgCBrsZv1gPDFn3mI3TwapJgTR0J4GAg
wXB/9GRHuCCTz7qvfQexBOwOt25OKVOhcvNFVI8bxV0hFl58Nlo56Qzt33X/JS32
jH2jIlz77pal1t5ZhnXJwCSWQyWsLnr5GtoxDisvvOl1o3Ey/WIllvCe8x7M+PvA
0/DIK/5niTSCwcv0LVCPIWsE/HCjsSWfdhnhtTnu1088OTKwb2dsa7wyBJItZUzw
fCTcmcGclViGUa2QAnXNFiVPj1y0PhFxAPMCU6mWPerCSH6cYn5neicsp2AYovoj
dRcs4LGrGf0S7PVJBq/DQdb44XbzvFkkp2SjS9WAnLpBv7RwP4bWfDvMCJsZWJOU
8J2r4ZbkVUjByQ3mAXMZN7bKC6hHBQLLzAwodloAV0OWHJ+Io96flTclDRPt4N6e
J8olEQezDKcgkZDg0GV8I9WxUzeTHI+QvnZxUzwsT/sJUPgxjSDjHlous5HU29ay
6lynoEjVFJd4yYAwh6gaRPMw34xKFT6a62D6bDmcL0MqPCpbcbOny+kgx0k7bzl5
FNsapJ5vCIaG0/tPTuWEY/jaqmhNNznXDr+sEX5l8Sk1ZQz8+/U=
=Y8qg
-----END PGP SIGNATURE-----
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
The CMS settings are very nearly what we use after tons of load testing we
changed newratio to 2 and it cut the 10 second pauses way down for us ....
huge heap though
On Wed, Sep 26, 2018, 2:17 PM Shawn Heisey <ap...@elyograg.org> wrote:
> On 9/26/2018 9:35 AM, Jeff Courtade wrote:
> > My concern with using g1 is solely based on finding this.
> > Does anyone have any information on this?
> >
> >
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
>
> I have never had a single problem with Solr running with the G1
> collector. I'm only aware of one actual bug in Lucene that mentions G1
> ... and it is specific to the 32-bit version of Java. It is strongly
> recommended for other reasons to only use a 64-bit Java.
>
> On the subject of the blog post mentioned by Zisis T... generally
> speaking, it is not a good idea to explicitly set the size of the
> various generations. G1 will tune the sizes of each generation as it
> runs for best results. By setting or limiting the size, that tuning
> cannot work with freedom, and you might be unhappy with the results.
>
> Here is a wiki page that contains my own experiments with garbage
> collection tuning:
>
> https://wiki.apache.org/solr/ShawnHeisey
>
> Thanks,
> Shawn
>
>
Re: Java version 11 for solr 7.5?
Posted by Erick Erickson <er...@gmail.com>.
bq. Can you tell me where I could get insight into the testing cycles
and results?
Hmmm, here's one example:
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22921/
But we're not really tracking JDK 11 separately at the moment. I'd
monitor the discussion at SOLR-12809 for the current state.
Do note that Uwe just installed the GA release of Java 11 on the test
machines in the last day or so. Perhaps many of the problems will go
away.
Best,
ErickOn Thu, Sep 27, 2018 at 12:34 AM Ere Maijala
<er...@helsinki.fi> wrote:
>
> Shawn Heisey kirjoitti 26.9.2018 klo 21.16:
> > On 9/26/2018 9:35 AM, Jeff Courtade wrote:
> >> My concern with using g1 is solely based on finding this.
> >> Does anyone have any information on this?
> >>
> >> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
> >>
> >
> > I have never had a single problem with Solr running with the G1
> > collector. I'm only aware of one actual bug in Lucene that mentions G1
> > ... and it is specific to the 32-bit version of Java. It is strongly
> > recommended for other reasons to only use a 64-bit Java.
> >
> > On the subject of the blog post mentioned by Zisis T... generally
> > speaking, it is not a good idea to explicitly set the size of the
> > various generations. G1 will tune the sizes of each generation as it
> > runs for best results. By setting or limiting the size, that tuning
> > cannot work with freedom, and you might be unhappy with the results.
> >
> > Here is a wiki page that contains my own experiments with garbage
> > collection tuning:
> >
> > https://wiki.apache.org/solr/ShawnHeisey
>
> There's one caveat that I know of: You might need to modify G1 tuning
> parameters if your Solr has dramatically changing usage patterns. For
> instance heavy search use during the day and heavy indexing work during
> the night. This may lead G1 to optimize too heavily for one and it
> taking a while to adjust for the other. At some point I've used e.g. the
> following settings to limit the young generation:
>
> -XX:+UnlockExperimentalVMOptions -XX:G1MaxNewSizePercent=5
>
> This was used with a very specific usage pattern, so it probably doesn't
> apply in most situations.
>
> Regards,
> Ere
>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
Re: Java version 11 for solr 7.5?
Posted by Ere Maijala <er...@helsinki.fi>.
Shawn Heisey kirjoitti 26.9.2018 klo 21.16:
> On 9/26/2018 9:35 AM, Jeff Courtade wrote:
>> My concern with using g1 is solely based on finding this.
>> Does anyone have any information on this?
>>
>> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
>>
>
> I have never had a single problem with Solr running with the G1
> collector. I'm only aware of one actual bug in Lucene that mentions G1
> ... and it is specific to the 32-bit version of Java. It is strongly
> recommended for other reasons to only use a 64-bit Java.
>
> On the subject of the blog post mentioned by Zisis T... generally
> speaking, it is not a good idea to explicitly set the size of the
> various generations. G1 will tune the sizes of each generation as it
> runs for best results. By setting or limiting the size, that tuning
> cannot work with freedom, and you might be unhappy with the results.
>
> Here is a wiki page that contains my own experiments with garbage
> collection tuning:
>
> https://wiki.apache.org/solr/ShawnHeisey
There's one caveat that I know of: You might need to modify G1 tuning
parameters if your Solr has dramatically changing usage patterns. For
instance heavy search use during the day and heavy indexing work during
the night. This may lead G1 to optimize too heavily for one and it
taking a while to adjust for the other. At some point I've used e.g. the
following settings to limit the young generation:
-XX:+UnlockExperimentalVMOptions -XX:G1MaxNewSizePercent=5
This was used with a very specific usage pattern, so it probably doesn't
apply in most situations.
Regards,
Ere
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Java version 11 for solr 7.5?
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/26/2018 9:35 AM, Jeff Courtade wrote:
> My concern with using g1 is solely based on finding this.
> Does anyone have any information on this?
>
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
I have never had a single problem with Solr running with the G1
collector. I'm only aware of one actual bug in Lucene that mentions G1
... and it is specific to the 32-bit version of Java. It is strongly
recommended for other reasons to only use a 64-bit Java.
On the subject of the blog post mentioned by Zisis T... generally
speaking, it is not a good idea to explicitly set the size of the
various generations. G1 will tune the sizes of each generation as it
runs for best results. By setting or limiting the size, that tuning
cannot work with freedom, and you might be unhappy with the results.
Here is a wiki page that contains my own experiments with garbage
collection tuning:
https://wiki.apache.org/solr/ShawnHeisey
Thanks,
Shawn
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
My concern with using g1 is solely based on finding this.
Does anyone have any information on this?
https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
"Do not, under any circumstances, run Lucene with the G1 garbage collector.
Lucene's test suite fails with the G1 garbage collector on a regular basis,
including bugs that cause index corruption. There is no person on this
planet that seems to understand such bugs (see
https://bugs.openjdk.java.net/browse/JDK-8038348, open for over a year), so
don't count on the situation changing soon. This information is not out of
date, and don't think that the next oracle java release will fix the
situation."
On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood <wu...@wunderwood.org>
wrote:
> We’ve been running G1 in prod for at least 18 months. Our biggest cluster
> is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on our
> 4.10.4 master/slave cluster.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Sep 26, 2018, at 7:37 AM, Jeff Courtade <co...@gmail.com>
> wrote:
> >
> > Thanks for that...
> > I am just starting to look at this I was unaware of the license debacle.
> >
> > Automated testing up to 10 is great.
> >
> > I am still curious about the GC1 being supported now...
> >
> > On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com> wrote:
> >
> >> Jeff Courtade wrote
> >>> Can we use GC1 garbage collection yet or do we still need to use CMS?
> >>
> >> I believe you should be safe to go with G1. We've applied it in in a
> Solr
> >> 6.6 cluster with 10 shards, 3 replicas per shard and an index of about
> >> 500GB
> >> (1,5T counting all replicas) and it works extremely well (throughput >
> >> 99%).
> >> The use-case includes complex search queries and faceting.
> >> There is also this post you can use as a starting point
> >>
> >>
> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
> >>
> >>
> >>
> >>
> >> --
> >> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >>
> > --
> >
> > Jeff Courtade
> > M: 240.507.6116 <(240)%20507-6116>
>
> --
Jeff Courtade
M: 240.507.6116
Re: Java version 11 for solr 7.5?
Posted by Walter Underwood <wu...@wunderwood.org>.
We’ve been running G1 in prod for at least 18 months. Our biggest cluster
is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on our
4.10.4 master/slave cluster.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Sep 26, 2018, at 7:37 AM, Jeff Courtade <co...@gmail.com> wrote:
>
> Thanks for that...
> I am just starting to look at this I was unaware of the license debacle.
>
> Automated testing up to 10 is great.
>
> I am still curious about the GC1 being supported now...
>
> On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com> wrote:
>
>> Jeff Courtade wrote
>>> Can we use GC1 garbage collection yet or do we still need to use CMS?
>>
>> I believe you should be safe to go with G1. We've applied it in in a Solr
>> 6.6 cluster with 10 shards, 3 replicas per shard and an index of about
>> 500GB
>> (1,5T counting all replicas) and it works extremely well (throughput >
>> 99%).
>> The use-case includes complex search queries and faceting.
>> There is also this post you can use as a starting point
>>
>> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
>>
>>
>>
>>
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>
> --
>
> Jeff Courtade
> M: 240.507.6116
Re: Java version 11 for solr 7.5?
Posted by Jeff Courtade <co...@gmail.com>.
Thanks for that...
I am just starting to look at this I was unaware of the license debacle.
Automated testing up to 10 is great.
I am still curious about the GC1 being supported now...
On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com> wrote:
> Jeff Courtade wrote
> > Can we use GC1 garbage collection yet or do we still need to use CMS?
>
> I believe you should be safe to go with G1. We've applied it in in a Solr
> 6.6 cluster with 10 shards, 3 replicas per shard and an index of about
> 500GB
> (1,5T counting all replicas) and it works extremely well (throughput >
> 99%).
> The use-case includes complex search queries and faceting.
> There is also this post you can use as a starting point
>
> http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
--
Jeff Courtade
M: 240.507.6116
Re: Java version 11 for solr 7.5?
Posted by "Zisis T." <zi...@runbox.com>.
Jeff Courtade wrote
> Can we use GC1 garbage collection yet or do we still need to use CMS?
I believe you should be safe to go with G1. We've applied it in in a Solr
6.6 cluster with 10 shards, 3 replicas per shard and an index of about 500GB
(1,5T counting all replicas) and it works extremely well (throughput > 99%).
The use-case includes complex search queries and faceting.
There is also this post you can use as a starting point
http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
RE: Java version 11 for solr 7.5?
Posted by Markus Jelsma <ma...@openindex.io>.
Indeed, but JDK-8038348 has been fixed very recently for Java 9 or higher.
-----Original message-----
> From:Jeff Courtade <co...@gmail.com>
> Sent: Wednesday 26th September 2018 17:36
> To: solr-user@lucene.apache.org
> Subject: Re: Java version 11 for solr 7.5?
>
> My concern with using g1 is solely based on finding this.
> Does anyone have any information on this?
>
> https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
>
> "Do not, under any circumstances, run Lucene with the G1 garbage collector.
> Lucene's test suite fails with the G1 garbage collector on a regular basis,
> including bugs that cause index corruption. There is no person on this
> planet that seems to understand such bugs (see
> https://bugs.openjdk.java.net/browse/JDK-8038348, open for over a year), so
> don't count on the situation changing soon. This information is not out of
> date, and don't think that the next oracle java release will fix the
> situation."
>
>
> On Wed, Sep 26, 2018 at 11:08 AM Walter Underwood <wu...@wunderwood.org>
> wrote:
>
> > We’ve been running G1 in prod for at least 18 months. Our biggest cluster
> > is 48 machines, each with 36 CPUs, running 6.6.2. We also run it on our
> > 4.10.4 master/slave cluster.
> >
> > wunder
> > Walter Underwood
> > wunder@wunderwood.org
> > http://observer.wunderwood.org/ (my blog)
> >
> > > On Sep 26, 2018, at 7:37 AM, Jeff Courtade <co...@gmail.com>
> > wrote:
> > >
> > > Thanks for that...
> > > I am just starting to look at this I was unaware of the license debacle.
> > >
> > > Automated testing up to 10 is great.
> > >
> > > I am still curious about the GC1 being supported now...
> > >
> > > On Wed, Sep 26, 2018 at 10:25 AM Zisis T. <zi...@runbox.com> wrote:
> > >
> > >> Jeff Courtade wrote
> > >>> Can we use GC1 garbage collection yet or do we still need to use CMS?
> > >>
> > >> I believe you should be safe to go with G1. We've applied it in in a
> > Solr
> > >> 6.6 cluster with 10 shards, 3 replicas per shard and an index of about
> > >> 500GB
> > >> (1,5T counting all replicas) and it works extremely well (throughput >
> > >> 99%).
> > >> The use-case includes complex search queries and faceting.
> > >> There is also this post you can use as a starting point
> > >>
> > >>
> > http://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > >>
> > > --
> > >
> > > Jeff Courtade
> > > M: 240.507.6116 <(240)%20507-6116>
> >
> > --
>
> Jeff Courtade
> M: 240.507.6116
>
Re: Java version 11 for solr 7.5?
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
The minimal required as per the changes file is 1.8 (8.x). I believe
there is an automated testing up to 10 and there were some issues
related to the modules but AFAIK they were resolved. So, you may be ok
until then.
However, the java 11 is a bit of a different beast. Both because the
public version has just been released, but also because of the license
change issues. You can see the article at:
https://blog.joda.org/2018/09/do-not-fall-into-oracles-java-11-trap.html
and the technical discussion at:
https://news.ycombinator.com/item?id=18074727
Not sure about GC, somebody else may comment on that.
Regards,
Alex.
On 26 September 2018 at 10:00, Jeff Courtade <co...@gmail.com> wrote:
> Hello,
>
> We are looking to migrate to solr 7.5 with java 11 from solr 4.3.0 with
> java 7.
>
> I have a couple basic questions...
>
> What version of Java is current solr 7.5 development and testing based on?
>
> Can we use java 11 with solr 7.5? any known issues?
>
> Can we use GC1 garbage collection yet or do we still need to use CMS?
>
> --
> Thanks,
>
> Jeff Courtade
> M: 240.507.6116