You are viewing a plain text version of this content. The canonical link for it is here.

Posted to announce@apache.org by Uwe Schindler <us...@apache.org> on 2011/07/28 23:13:36 UTC

[WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler
optimizations, which miscompile some loops. This can affect code of several
Apache projects. Sometimes JVMs only crash, but in several cases, results
calculated can be incorrect, leading to bugs in applications (see Hotspot
bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are
affected by these bugs, namely all versions released until today. Solr users
with the default configuration will have Java crashing with SIGSEGV as soon
as they start to index documents, as one affected part is the well-known
Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
miscompiled, too, leading to index corruption (especially on Lucene trunk
with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release,
so Oracle had no time to fix those bugs, affecting also many more
applications. In response to our questions, they proposed to include the
fixes into service release u2 (eventually into service release u1, see [6]).
This means you cannot use Apache Lucene/Solr with Java 7 releases before
Update 2! If you do, please don't open bug reports, it is not the
committers' fault! At least disable loop optimizations using the
-XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the
unicode version shipped with Java 7 changed and tokenization behaves
differently (e.g. lowercasing). For more information, read
JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project,
Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
[4] https://issues.apache.org/jira/browse/LUCENE-3335
[5] https://issues.apache.org/jira/browse/LUCENE-3346
[6] http://s.apache.org/StQ

-----
Uwe Schindler
uschindler@apache.org 
Apache Lucene PMC Member / Committer
Bremen, Germany
http://lucene.apache.org/



--------------------------------------------------------------------- 
To unsubscribe, e-mail: announce-unsubscribe@apache.org 
For additional commands, e-mail: announce-help@apache.org

RE: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Uwe Schindler <uw...@thetaphi.de>.

Hi Sanne,

Thanks, we already have the required patches installed on the Jenkins build server for testing purposes. In fact it’s a minimum of three bugs. It fixes all those bugs reported in the issues.

The patch we have installed can be found on LUCENE-3335:
https://issues.apache.org/jira/secure/attachment/12487824/patch-0uwe.patch

Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Sanne Grinovero [mailto:sanne.grinovero@gmail.com]
> Sent: Friday, July 29, 2011 11:11 AM
> To: java-user@lucene.apache.org
> Cc: general@lucene.apache.org; solr-user@lucene.apache.org
> Subject: Re: [WARNING] Index corruption and crashes in Apache Lucene
> Core / Apache Solr with Java 7
> 
> Hello,
> thanks for the warning, that's a pretty nasty bug.
> 
> A patch was made for OpenJDK, if anybody is interested to try it out that
> would be great:
> http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/rev/4e761e7e6e12
> 
> Regards,
> Sanne
> 
> 2011/7/28 Uwe Schindler <us...@apache.org>:
> > Hello Apache Lucene & Apache Solr users, Hello users of other
> > Java-based Apache projects,
> >
> > Oracle released Java 7 today. Unfortunately it contains hotspot
> > compiler optimizations, which miscompile some loops. This can affect
> > code of several Apache projects. Sometimes JVMs only crash, but in
> > several cases, results calculated can be incorrect, leading to bugs in
> > applications (see Hotspot bugs 7070134 [1], 7044738 [2], 7068051 [3]).
> >
> > Apache Lucene Core and Apache Solr are two Apache projects, which are
> > affected by these bugs, namely all versions released until today. Solr
> > users with the default configuration will have Java crashing with
> > SIGSEGV as soon as they start to index documents, as one affected part
> > is the well-known Porter stemmer (see LUCENE-3335 [4]). Other loops in
> > Lucene may be miscompiled, too, leading to index corruption
> > (especially on Lucene trunk with pulsing codec; other loops may be
> affected, too - LUCENE-3346 [5]).
> >
> > These problems were detected only 5 days before the official Java 7
> > release, so Oracle had no time to fix those bugs, affecting also many
> > more applications. In response to our questions, they proposed to
> > include the fixes into service release u2 (eventually into service release u1,
> see [6]).
> > This means you cannot use Apache Lucene/Solr with Java 7 releases
> > before Update 2! If you do, please don't open bug reports, it is not
> > the committers' fault! At least disable loop optimizations using the
> > -XX:-UseLoopPredicate JVM option to not risk index corruptions.
> >
> > Please note: Also Java 6 users are affected, if they use one of those
> > JVM options, which are not enabled by default:
> > -XX:+OptimizeStringConcat or -XX:+AggressiveOpts
> >
> > It is strongly recommended not to use any hotspot optimization
> > switches in any Java version without extensive testing!
> >
> > In case you upgrade to Java 7, remember that you may have to reindex,
> > as the unicode version shipped with Java 7 changed and tokenization
> > behaves differently (e.g. lowercasing). For more information, read
> > JRE_VERSION_MIGRATION.txt in your distribution package!
> >
> > On behalf of the Lucene project,
> > Uwe
> >
> > [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> > [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> > [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> > [4] https://issues.apache.org/jira/browse/LUCENE-3335
> > [5] https://issues.apache.org/jira/browse/LUCENE-3346
> > [6] http://s.apache.org/StQ
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > Apache Lucene PMC Member / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Sanne Grinovero <sa...@gmail.com>.

Hello,
thanks for the warning, that's a pretty nasty bug.

A patch was made for OpenJDK, if anybody is interested to try it out
that would be great:
http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/rev/4e761e7e6e12

Regards,
Sanne

2011/7/28 Uwe Schindler <us...@apache.org>:
> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
>
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
>
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
>
> These problems were detected only 5 days before the official Java 7 release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
>
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
>
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
>
> In case you upgrade to Java 7, remember that you may have to reindex, as the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
>
> On behalf of the Lucene project,
> Uwe
>
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
>
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Sanne Grinovero <sa...@gmail.com>.

Hello,
thanks for the warning, that's a pretty nasty bug.

A patch was made for OpenJDK, if anybody is interested to try it out
that would be great:
http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/rev/4e761e7e6e12

Regards,
Sanne

2011/7/28 Uwe Schindler <us...@apache.org>:
> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
>
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
>
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
>
> These problems were detected only 5 days before the official Java 7 release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
>
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
>
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
>
> In case you upgrade to Java 7, remember that you may have to reindex, as the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
>
> On behalf of the Lucene project,
> Uwe
>
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
>
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Sanne Grinovero <sa...@gmail.com>.

Hello,
thanks for the warning, that's a pretty nasty bug.

A patch was made for OpenJDK, if anybody is interested to try it out
that would be great:
http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/rev/4e761e7e6e12

Regards,
Sanne

2011/7/28 Uwe Schindler <us...@apache.org>:
> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
>
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
>
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
>
> These problems were detected only 5 days before the official Java 7 release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
>
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
>
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
>
> In case you upgrade to Java 7, remember that you may have to reindex, as the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
>
> On behalf of the Lucene project,
> Uwe
>
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
>
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Dave Fisher <da...@comcast.net>.

Be warned about Java 7 u1 and Hotspot.

Regards,
Dave

Begin forwarded message:

> From: Konstantin Kolinko <kn...@gmail.com>
> Date: July 29, 2011 7:35:42 AM PDT
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7
> Reply-To: "Tomcat Users List" <us...@tomcat.apache.org>
> 
> Forwarding from announce at apache org. I think it is better to be warned.
> 
> Best regards,
> Konstantin Kolinko
> 
> ---------- Forwarded message ----------
> From: Uwe Schindler
> Date: 2011/7/29
> Subject: [WARNING] Index corruption and crashes in Apache Lucene Core
> / Apache Solr with Java 7
> 
> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
> 
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
> 
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
> 
> These problems were detected only 5 days before the official Java 7 release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
> 
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
> 
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
> 
> In case you upgrade to Java 7, remember that you may have to reindex, as the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
> 
> On behalf of the Lucene project,
> Uwe
> 
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
> 
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org

Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Dave Fisher <da...@comcast.net>.

Be warned about Java 7 u1 and Hotspot.

Regards,
Dave

Begin forwarded message:

> From: Konstantin Kolinko <kn...@gmail.com>
> Date: July 29, 2011 7:35:42 AM PDT
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7
> Reply-To: "Tomcat Users List" <us...@tomcat.apache.org>
> 
> Forwarding from announce at apache org. I think it is better to be warned.
> 
> Best regards,
> Konstantin Kolinko
> 
> ---------- Forwarded message ----------
> From: Uwe Schindler
> Date: 2011/7/29
> Subject: [WARNING] Index corruption and crashes in Apache Lucene Core
> / Apache Solr with Java 7
> 
> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
> 
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
> 
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
> 
> These problems were detected only 5 days before the official Java 7 release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
> 
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
> 
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
> 
> In case you upgrade to Java 7, remember that you may have to reindex, as the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
> 
> On behalf of the Lucene project,
> Uwe
> 
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
> 
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org

Re: Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by André Warnier <aw...@ice-sa.com>.

Many thanks to Konstantin (and Uwe Schindler) for broadcasting this.
It will probably save a lot of time to a lot of people to know this in advance.

Konstantin Kolinko wrote:
> Forwarding from announce at apache org. I think it is better to be warned.
> 
> Best regards,
> Konstantin Kolinko
> 
...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Konstantin Kolinko <kn...@gmail.com>.

Forwarding from announce at apache org. I think it is better to be warned.

Best regards,
Konstantin Kolinko

---------- Forwarded message ----------
From: Uwe Schindler
Date: 2011/7/29
Subject: [WARNING] Index corruption and crashes in Apache Lucene Core
/ Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler
optimizations, which miscompile some loops. This can affect code of several
Apache projects. Sometimes JVMs only crash, but in several cases, results
calculated can be incorrect, leading to bugs in applications (see Hotspot
bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are
affected by these bugs, namely all versions released until today. Solr users
with the default configuration will have Java crashing with SIGSEGV as soon
as they start to index documents, as one affected part is the well-known
Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
miscompiled, too, leading to index corruption (especially on Lucene trunk
with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release,
so Oracle had no time to fix those bugs, affecting also many more
applications. In response to our questions, they proposed to include the
fixes into service release u2 (eventually into service release u1, see [6]).
This means you cannot use Apache Lucene/Solr with Java 7 releases before
Update 2! If you do, please don't open bug reports, it is not the
committers' fault! At least disable loop optimizations using the
-XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the
unicode version shipped with Java 7 changed and tokenization behaves
differently (e.g. lowercasing). For more information, read
JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project,
Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
[4] https://issues.apache.org/jira/browse/LUCENE-3335
[5] https://issues.apache.org/jira/browse/LUCENE-3346
[6] http://s.apache.org/StQ

-----
Uwe Schindler
uschindler@apache.org
Apache Lucene PMC Member / Committer
Bremen, Germany
http://lucene.apache.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Oleg Tikhonov <ol...@gmail.com>.

FYI,

On Fri, Jul 29, 2011 at 12:13 AM, Uwe Schindler <us...@apache.org>wrote:

> Hello Apache Lucene & Apache Solr users,
> Hello users of other Java-based Apache projects,
>
> Oracle released Java 7 today. Unfortunately it contains hotspot compiler
> optimizations, which miscompile some loops. This can affect code of several
> Apache projects. Sometimes JVMs only crash, but in several cases, results
> calculated can be incorrect, leading to bugs in applications (see Hotspot
> bugs 7070134 [1], 7044738 [2], 7068051 [3]).
>
> Apache Lucene Core and Apache Solr are two Apache projects, which are
> affected by these bugs, namely all versions released until today. Solr
> users
> with the default configuration will have Java crashing with SIGSEGV as soon
> as they start to index documents, as one affected part is the well-known
> Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
> miscompiled, too, leading to index corruption (especially on Lucene trunk
> with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).
>
> These problems were detected only 5 days before the official Java 7
> release,
> so Oracle had no time to fix those bugs, affecting also many more
> applications. In response to our questions, they proposed to include the
> fixes into service release u2 (eventually into service release u1, see
> [6]).
> This means you cannot use Apache Lucene/Solr with Java 7 releases before
> Update 2! If you do, please don't open bug reports, it is not the
> committers' fault! At least disable loop optimizations using the
> -XX:-UseLoopPredicate JVM option to not risk index corruptions.
>
> Please note: Also Java 6 users are affected, if they use one of those JVM
> options, which are not enabled by default: -XX:+OptimizeStringConcat or
> -XX:+AggressiveOpts
>
> It is strongly recommended not to use any hotspot optimization switches in
> any Java version without extensive testing!
>
> In case you upgrade to Java 7, remember that you may have to reindex, as
> the
> unicode version shipped with Java 7 changed and tokenization behaves
> differently (e.g. lowercasing). For more information, read
> JRE_VERSION_MIGRATION.txt in your distribution package!
>
> On behalf of the Lucene project,
> Uwe
>
> [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
> [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
> [4] https://issues.apache.org/jira/browse/LUCENE-3335
> [5] https://issues.apache.org/jira/browse/LUCENE-3346
> [6] http://s.apache.org/StQ
>
> -----
> Uwe Schindler
> uschindler@apache.org
> Apache Lucene PMC Member / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>
>

Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Florian Holeczek <fl...@holeczek.de>.

Dear JSPWiki users,

Java 7 has been released and seems to contain some serious bugs, see the forwarded message below.
Since JSPWiki is using Lucene, too, you shouldn't run it with Java 7 in production environments right now.

Best regards
Florian Holeczek

----- Weitergeleitete Mail -----
Von: "Uwe Schindler" <us...@apache.org>
An: announce@apache.org, general@lucene.apache.org, java-user@lucene.apache.org, solr-user@lucene.apache.org
Gesendet: Donnerstag, 28. Juli 2011 23:13:36
Betreff: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler
optimizations, which miscompile some loops. This can affect code of several
Apache projects. Sometimes JVMs only crash, but in several cases, results
calculated can be incorrect, leading to bugs in applications (see Hotspot
bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are
affected by these bugs, namely all versions released until today. Solr users
with the default configuration will have Java crashing with SIGSEGV as soon
as they start to index documents, as one affected part is the well-known
Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
miscompiled, too, leading to index corruption (especially on Lucene trunk
with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release,
so Oracle had no time to fix those bugs, affecting also many more
applications. In response to our questions, they proposed to include the
fixes into service release u2 (eventually into service release u1, see [6]).
This means you cannot use Apache Lucene/Solr with Java 7 releases before
Update 2! If you do, please don't open bug reports, it is not the
committers' fault! At least disable loop optimizations using the
-XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the
unicode version shipped with Java 7 changed and tokenization behaves
differently (e.g. lowercasing). For more information, read
JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project,
Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
[4] https://issues.apache.org/jira/browse/LUCENE-3335
[5] https://issues.apache.org/jira/browse/LUCENE-3346
[6] http://s.apache.org/StQ

-----
Uwe Schindler
uschindler@apache.org
Apache Lucene PMC Member / Committer
Bremen, Germany
http://lucene.apache.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: announce-unsubscribe@apache.org
For additional commands, e-mail: announce-help@apache.org

Fwd: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Posted by Florian Holeczek <fl...@holeczek.de>.

Dear JSPWiki users,

Best regards
Florian Holeczek

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

On behalf of the Lucene project,
Uwe

-----
Uwe Schindler
uschindler@apache.org
Apache Lucene PMC Member / Committer
Bremen, Germany
http://lucene.apache.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: announce-unsubscribe@apache.org
For additional commands, e-mail: announce-help@apache.org