You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Uwe Schindler <uw...@thetaphi.de> on 2014/08/17 23:08:34 UTC
Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Hi,
it took some time to find out, why sometimes Lucene/Solr's tests failed with a very strange error message when trying to fork a process on MacOSX and Linux. The reason for this bug to appear is our randomized testing framework: As you know, we execute all tests in different settings like default charsets, default locales or default timezones, to find bugs caused by settings applied to the environment, that may affect the code in an incorrect way. For more information, see this (already famous) blog entry: http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html
In our case the JDK issue was:
https://bugs.openjdk.java.net/browse/JDK-8047340
The issue on our side:
https://issues.apache.org/jira/browse/SOLR-6387
We just want to add, that this bug also applies to Java 7 - the same issue exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH / Locale.ROOT) should be backported to Java 7 and Java 6 for customers with support contract, too. As I cannot modify the bug report, it would be good to add issues for Java 7 and Java 6, too.
Uwe
PS: We use the Forbidden-APIs checker (https://code.google.com/p/forbidden-apis/) to detect such bugs in Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other projects I don't know of yet. Maybe when building the rt.jar, you should do the same :-) Calling String#toLowerCase() without an explicit Locale is a bug, if you rely on locale-insensitive behavior.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Balchandra Vaidya <ba...@oracle.com>.
Hi Uwe,
Here is the JBS ID for jdk 7.
https://bugs.openjdk.java.net/browse/JDK-8055301
Thanks
Balchandra
On 08/18/14 03:31 PM, Balchandra Vaidya wrote:
>
> Hi Uwe,
>
> Thank you for the clarification. I have updated the bug
> and will let you the status of the backport.
>
> Kind regards
> Balchandra
>
>
>
>
>
> On 08/18/14 03:01 PM, Uwe Schindler wrote:
>>
>> Hi Balchandra,
>>
>> the test on MacOSX was actually done with Oracle’s JDK: 7u65. I just
>> added that Apple’s own JDK6 on OSX does not have this bug, which does
>> not mean Oracle’s JDK6 on Solaris or BSD does not have it.
>>
>> On Linux, this bug does not happen, because it uses another fork
>> implementation, without a “i” in it (so no Turkish lowercasing
>> problem): LaunchMechanism.VFORK (Linux)<->
>> LaunchMechanism.POS*I*X_SPAWN(BSD).
>>
>> Uwe
>>
>> -----
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de <http://www.thetaphi.de/>
>>
>> eMail: uwe@thetaphi.de
>>
>> *From:*Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
>> *Sent:* Monday, August 18, 2014 3:53 PM
>> *To:* Uwe Schindler
>> *Cc:* dev@lucene.apache.org; rory.odonnell@oracle.com; 'Dalibor Topic'
>> *Subject:* Re: Apache Lucene/Solr: Turkish lowercasing bug in
>> Runtime#exec()
>>
>>
>> Hi Uwe,
>>
>> I will update your comments in the jbs bug.
>>
>> Does this failure reproducible with Oracle JDK
>> <http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html>?
>> I could not
>> reproduce it on Linux (OL6 u5) and Solaris 11 so far. I will
>> try it on a Mac later.
>>
>>
>> Thanks
>> Balchandra
>>
>>
>> On 08/18/14 01:48 PM, Uwe Schindler wrote:
>>
>> Hi Balchandra:
>>
>>
>>
>> I checked it out with Java 7 on FreeBSD:
>>
>>
>>
>> ==== SNIP ====
>>
>> [uschindler@lucene ~]$ cat Test.java
>>
>> import java.util.Locale;
>>
>>
>>
>> public class Test {
>>
>>
>>
>> public static void main(final String[] args) throws Exception {
>>
>> Locale.setDefault(new Locale("tr", ""));
>>
>> Process exec = Runtime.getRuntime().exec("ls");
>>
>> }
>>
>>
>>
>> }
>>
>>
>>
>> [uschindler@lucene ~]$ javac Test.java
>>
>>
>>
>> [uschindler@lucene ~]$ java -version
>>
>> openjdk version "1.7.0_60"
>>
>> OpenJDK Runtime Environment (build 1.7.0_60-b19)
>>
>> OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
>>
>> [uschindler@lucene ~]$ java Test
>>
>> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
>>
>> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>>
>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
>>
>> at java.lang.Runtime.exec(Runtime.java:617)
>>
>> at java.lang.Runtime.exec(Runtime.java:450)
>>
>> at java.lang.Runtime.exec(Runtime.java:347)
>>
>> at Test.main(Test.java:7)
>>
>> ==== SNIP ====
>>
>>
>>
>> On MacOSX:
>>
>>
>>
>> ==== SNIP ====
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/javac Test.java
>>
>>
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java -version
>>
>> java version "1.6.0_65"
>>
>> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>>
>> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java Test
>>
>>
>>
>> -> So it seems that Apple's JDK6 is immune. Unfortunately I don’t have a FreeBSD version available. I also have no Solaris or AIX one.
>>
>>
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java -version
>>
>> java version "1.7.0_65"
>>
>> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
>>
>> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java Test
>>
>> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:104)
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
>>
>> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>>
>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
>>
>> at java.lang.Runtime.exec(Runtime.java:617)
>>
>> at java.lang.Runtime.exec(Runtime.java:450)
>>
>> at java.lang.Runtime.exec(Runtime.java:347)
>>
>> at Test.main(Test.java:7)
>>
>>
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java -version
>>
>> java version "1.8.0_11"
>>
>> Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
>>
>> Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)
>>
>> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java Test
>>
>> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:105)
>>
>> at java.lang.UNIXProcess$1.run(UNIXProcess.java:94)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:92)
>>
>> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>>
>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
>>
>> at java.lang.Runtime.exec(Runtime.java:620)
>>
>> at java.lang.Runtime.exec(Runtime.java:450)
>>
>> at java.lang.Runtime.exec(Runtime.java:347)
>>
>> at Test.main(Test.java:7)
>>
>> ==== SNIP ====
>>
>>
>>
>> Uwe
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>>
>>
>>
>>
>>
>> -----Original Message-----
>>
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>>
>> Sent: Monday, August 18, 2014 1:02 PM
>>
>> To: 'Balchandra Vaidya'
>>
>> Cc:rory.odonnell@oracle.com <ma...@oracle.com>; 'Dalibor Topic';dev@lucene.apache.org <ma...@lucene.apache.org>
>>
>> Subject: RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
>>
>>
>>
>> Hi Balchandra,
>>
>>
>>
>> it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
>>
>> + /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
>>
>> openjdk version "1.7.0_60"
>>
>> OpenJDK Runtime Environment (build 1.7.0_60-b19) OpenJDK 64-Bit Server
>>
>> VM (build 24.60-b09, mixed mode)
>>
>>
>>
>> and MacOSX (JDK 7u65):
>>
>> + /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java
>>
>> + -XX:+UseCompressedOops -XX:+UseG1GC -version
>>
>> java version "1.7.0_65"
>>
>> Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-
>>
>> Bit Server VM (build 24.65-b04, mixed mode)
>>
>>
>>
>> See the attached failure notices from the Jenkins Server.
>>
>>
>>
>> After lunch I will do a quick check on command line and 3-liner Java test; any
>>
>> MacOSX or BSD machine should be fine.
>>
>>
>>
>> Uwe
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>>
>>
>>
>> -----Original Message-----
>>
>> From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
>>
>> Sent: Monday, August 18, 2014 12:39 PM
>>
>> To: Uwe Schindler
>>
>> Cc:rory.odonnell@oracle.com <ma...@oracle.com>; 'Dalibor Topic';dev@lucene.apache.org <ma...@lucene.apache.org>
>>
>> Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in
>>
>> Runtime#exec()
>>
>>
>>
>>
>>
>> Hi Uwe,
>>
>>
>>
>> I have added your comments to JDK-8047340. The bug has an original
>>
>> comment "REGRESSION. Last worked in version 7u60"- are you sure the
>>
>> bug existed in JDK 7?
>>
>>
>>
>> Thanks
>>
>> Balchandra
>>
>>
>>
>>
>>
>>
>>
>> On 08/17/14 10:08 PM, Uwe Schindler wrote:
>>
>> Hi,
>>
>>
>>
>> it took some time to find out, why sometimes Lucene/Solr's tests
>>
>> failed with a very strange error message when trying to fork a
>>
>> process on MacOSX and Linux. The reason for this bug to appear is
>>
>> our randomized testing framework: As you know, we execute all tests
>>
>> in different settings like default charsets, default locales or
>>
>> default timezones, to find bugs caused by settings applied to the
>>
>> environment, that may affect the code in an incorrect way. For more
>>
>> information, see this (already famous) blog entry:
>>
>> http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and
>>
>> .h
>>
>> tml
>>
>>
>>
>> In our case the JDK issue was:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8047340
>>
>>
>>
>> The issue on our side:
>>
>> https://issues.apache.org/jira/browse/SOLR-6387
>>
>>
>>
>> We just want to add, that this bug also applies to Java 7 - the same
>>
>> issue
>>
>> exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH
>>
>> /
>>
>> Locale.ROOT) should be backported to Java 7 and Java 6 for customers
>>
>> with support contract, too. As I cannot modify the bug report, it
>>
>> would be good to add issues for Java 7 and Java 6, too.
>>
>>
>>
>> Uwe
>>
>>
>>
>> PS: We use the Forbidden-APIs checker
>>
>> (https://code.google.com/p/forbidden-apis/) to detect such bugs in
>>
>> Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other
>>
>> projects I don't know of yet. Maybe when building the rt.jar, you
>>
>> should do the same :-) Calling String#toLowerCase() without an
>>
>> explicit Locale is a bug, if you rely on locale-insensitive behavior.
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremenhttp://www.thetaphi.de
>>
>> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Balchandra Vaidya <ba...@oracle.com>.
Hi Uwe,
Thank you for the clarification. I have updated the bug
and will let you the status of the backport.
Kind regards
Balchandra
On 08/18/14 03:01 PM, Uwe Schindler wrote:
>
> Hi Balchandra,
>
> the test on MacOSX was actually done with Oracle’s JDK: 7u65. I just
> added that Apple’s own JDK6 on OSX does not have this bug, which does
> not mean Oracle’s JDK6 on Solaris or BSD does not have it.
>
> On Linux, this bug does not happen, because it uses another fork
> implementation, without a “i” in it (so no Turkish lowercasing
> problem): LaunchMechanism.VFORK (Linux)<->
> LaunchMechanism.POS*I*X_SPAWN(BSD).
>
> Uwe
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de <http://www.thetaphi.de/>
>
> eMail: uwe@thetaphi.de
>
> *From:*Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
> *Sent:* Monday, August 18, 2014 3:53 PM
> *To:* Uwe Schindler
> *Cc:* dev@lucene.apache.org; rory.odonnell@oracle.com; 'Dalibor Topic'
> *Subject:* Re: Apache Lucene/Solr: Turkish lowercasing bug in
> Runtime#exec()
>
>
> Hi Uwe,
>
> I will update your comments in the jbs bug.
>
> Does this failure reproducible with Oracle JDK
> <http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html>?
> I could not
> reproduce it on Linux (OL6 u5) and Solaris 11 so far. I will
> try it on a Mac later.
>
>
> Thanks
> Balchandra
>
>
> On 08/18/14 01:48 PM, Uwe Schindler wrote:
>
> Hi Balchandra:
>
>
>
> I checked it out with Java 7 on FreeBSD:
>
>
>
> ==== SNIP ====
>
> [uschindler@lucene ~]$ cat Test.java
>
> import java.util.Locale;
>
>
>
> public class Test {
>
>
>
> public static void main(final String[] args) throws Exception {
>
> Locale.setDefault(new Locale("tr", ""));
>
> Process exec = Runtime.getRuntime().exec("ls");
>
> }
>
>
>
> }
>
>
>
> [uschindler@lucene ~]$ javac Test.java
>
>
>
> [uschindler@lucene ~]$ java -version
>
> openjdk version "1.7.0_60"
>
> OpenJDK Runtime Environment (build 1.7.0_60-b19)
>
> OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
>
> [uschindler@lucene ~]$ java Test
>
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
>
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
>
> at java.lang.Runtime.exec(Runtime.java:617)
>
> at java.lang.Runtime.exec(Runtime.java:450)
>
> at java.lang.Runtime.exec(Runtime.java:347)
>
> at Test.main(Test.java:7)
>
> ==== SNIP ====
>
>
>
> On MacOSX:
>
>
>
> ==== SNIP ====
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/javac Test.java
>
>
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java -version
>
> java version "1.6.0_65"
>
> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>
> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java Test
>
>
>
> -> So it seems that Apple's JDK6 is immune. Unfortunately I don’t have a FreeBSD version available. I also have no Solaris or AIX one.
>
>
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java -version
>
> java version "1.7.0_65"
>
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
>
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java Test
>
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:104)
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
>
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
>
> at java.lang.Runtime.exec(Runtime.java:617)
>
> at java.lang.Runtime.exec(Runtime.java:450)
>
> at java.lang.Runtime.exec(Runtime.java:347)
>
> at Test.main(Test.java:7)
>
>
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java -version
>
> java version "1.8.0_11"
>
> Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
>
> Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java Test
>
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:105)
>
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:94)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:92)
>
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
>
> at java.lang.Runtime.exec(Runtime.java:620)
>
> at java.lang.Runtime.exec(Runtime.java:450)
>
> at java.lang.Runtime.exec(Runtime.java:347)
>
> at Test.main(Test.java:7)
>
> ==== SNIP ====
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>
>
>
>
>
> -----Original Message-----
>
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>
> Sent: Monday, August 18, 2014 1:02 PM
>
> To: 'Balchandra Vaidya'
>
> Cc:rory.odonnell@oracle.com <ma...@oracle.com>; 'Dalibor Topic';dev@lucene.apache.org <ma...@lucene.apache.org>
>
> Subject: RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
>
>
>
> Hi Balchandra,
>
>
>
> it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
>
> + /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
>
> openjdk version "1.7.0_60"
>
> OpenJDK Runtime Environment (build 1.7.0_60-b19) OpenJDK 64-Bit Server
>
> VM (build 24.60-b09, mixed mode)
>
>
>
> and MacOSX (JDK 7u65):
>
> + /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java
>
> + -XX:+UseCompressedOops -XX:+UseG1GC -version
>
> java version "1.7.0_65"
>
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-
>
> Bit Server VM (build 24.65-b04, mixed mode)
>
>
>
> See the attached failure notices from the Jenkins Server.
>
>
>
> After lunch I will do a quick check on command line and 3-liner Java test; any
>
> MacOSX or BSD machine should be fine.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>
>
>
> -----Original Message-----
>
> From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
>
> Sent: Monday, August 18, 2014 12:39 PM
>
> To: Uwe Schindler
>
> Cc:rory.odonnell@oracle.com <ma...@oracle.com>; 'Dalibor Topic';dev@lucene.apache.org <ma...@lucene.apache.org>
>
> Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in
>
> Runtime#exec()
>
>
>
>
>
> Hi Uwe,
>
>
>
> I have added your comments to JDK-8047340. The bug has an original
>
> comment "REGRESSION. Last worked in version 7u60"- are you sure the
>
> bug existed in JDK 7?
>
>
>
> Thanks
>
> Balchandra
>
>
>
>
>
>
>
> On 08/17/14 10:08 PM, Uwe Schindler wrote:
>
> Hi,
>
>
>
> it took some time to find out, why sometimes Lucene/Solr's tests
>
> failed with a very strange error message when trying to fork a
>
> process on MacOSX and Linux. The reason for this bug to appear is
>
> our randomized testing framework: As you know, we execute all tests
>
> in different settings like default charsets, default locales or
>
> default timezones, to find bugs caused by settings applied to the
>
> environment, that may affect the code in an incorrect way. For more
>
> information, see this (already famous) blog entry:
>
> http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and
>
> .h
>
> tml
>
>
>
> In our case the JDK issue was:
>
> https://bugs.openjdk.java.net/browse/JDK-8047340
>
>
>
> The issue on our side:
>
> https://issues.apache.org/jira/browse/SOLR-6387
>
>
>
> We just want to add, that this bug also applies to Java 7 - the same
>
> issue
>
> exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH
>
> /
>
> Locale.ROOT) should be backported to Java 7 and Java 6 for customers
>
> with support contract, too. As I cannot modify the bug report, it
>
> would be good to add issues for Java 7 and Java 6, too.
>
>
>
> Uwe
>
>
>
> PS: We use the Forbidden-APIs checker
>
> (https://code.google.com/p/forbidden-apis/) to detect such bugs in
>
> Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other
>
> projects I don't know of yet. Maybe when building the rt.jar, you
>
> should do the same :-) Calling String#toLowerCase() without an
>
> explicit Locale is a bug, if you rely on locale-insensitive behavior.
>
>
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremenhttp://www.thetaphi.de
>
> eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>
>
>
>
>
>
>
>
>
RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Balchandra,
the test on MacOSX was actually done with Oracle’s JDK: 7u65. I just added that Apple’s own JDK6 on OSX does not have this bug, which does not mean Oracle’s JDK6 on Solaris or BSD does not have it.
On Linux, this bug does not happen, because it uses another fork implementation, without a “i” in it (so no Turkish lowercasing problem): LaunchMechanism.VFORK (Linux)<-> LaunchMechanism.POSIX_SPAWN (BSD).
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
<http://www.thetaphi.de/> http://www.thetaphi.de
eMail: uwe@thetaphi.de
From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
Sent: Monday, August 18, 2014 3:53 PM
To: Uwe Schindler
Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; 'Dalibor Topic'
Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Hi Uwe,
I will update your comments in the jbs bug.
Does this failure reproducible with Oracle JDK <http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html> ? I could not
reproduce it on Linux (OL6 u5) and Solaris 11 so far. I will
try it on a Mac later.
Thanks
Balchandra
On 08/18/14 01:48 PM, Uwe Schindler wrote:
Hi Balchandra:
I checked it out with Java 7 on FreeBSD:
==== SNIP ====
[uschindler@lucene ~]$ cat Test.java
import java.util.Locale;
public class Test {
public static void main(final String[] args) throws Exception {
Locale.setDefault(new Locale("tr", ""));
Process exec = Runtime.getRuntime().exec("ls");
}
}
[uschindler@lucene ~]$ javac Test.java
[uschindler@lucene ~]$ java -version
openjdk version "1.7.0_60"
OpenJDK Runtime Environment (build 1.7.0_60-b19)
OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
[uschindler@lucene ~]$ java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
==== SNIP ====
On MacOSX:
==== SNIP ====
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/javac Test.java
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java -version
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java Test
-> So it seems that Apple's JDK6 is immune. Unfortunately I don’t have a FreeBSD version available. I also have no Solaris or AIX one.
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java -version
java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:104)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java -version
java version "1.8.0_11"
Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:105)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:94)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:92)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
at java.lang.Runtime.exec(Runtime.java:620)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
==== SNIP ====
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
-----Original Message-----
From: Uwe Schindler [mailto:uwe@thetaphi.de]
Sent: Monday, August 18, 2014 1:02 PM
To: 'Balchandra Vaidya'
Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
Subject: RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Hi Balchandra,
it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
+ /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
openjdk version "1.7.0_60"
OpenJDK Runtime Environment (build 1.7.0_60-b19) OpenJDK 64-Bit Server
VM (build 24.60-b09, mixed mode)
and MacOSX (JDK 7u65):
+ /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java
+ -XX:+UseCompressedOops -XX:+UseG1GC -version
java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-
Bit Server VM (build 24.65-b04, mixed mode)
See the attached failure notices from the Jenkins Server.
After lunch I will do a quick check on command line and 3-liner Java test; any
MacOSX or BSD machine should be fine.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
-----Original Message-----
From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
Sent: Monday, August 18, 2014 12:39 PM
To: Uwe Schindler
Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in
Runtime#exec()
Hi Uwe,
I have added your comments to JDK-8047340. The bug has an original
comment "REGRESSION. Last worked in version 7u60"- are you sure the
bug existed in JDK 7?
Thanks
Balchandra
On 08/17/14 10:08 PM, Uwe Schindler wrote:
Hi,
it took some time to find out, why sometimes Lucene/Solr's tests
failed with a very strange error message when trying to fork a
process on MacOSX and Linux. The reason for this bug to appear is
our randomized testing framework: As you know, we execute all tests
in different settings like default charsets, default locales or
default timezones, to find bugs caused by settings applied to the
environment, that may affect the code in an incorrect way. For more
information, see this (already famous) blog entry:
http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and
.h
tml
In our case the JDK issue was:
https://bugs.openjdk.java.net/browse/JDK-8047340
The issue on our side:
https://issues.apache.org/jira/browse/SOLR-6387
We just want to add, that this bug also applies to Java 7 - the same
issue
exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH
/
Locale.ROOT) should be backported to Java 7 and Java 6 for customers
with support contract, too. As I cannot modify the bug report, it
would be good to add issues for Java 7 and Java 6, too.
Uwe
PS: We use the Forbidden-APIs checker
(https://code.google.com/p/forbidden-apis/) to detect such bugs in
Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other
projects I don't know of yet. Maybe when building the rt.jar, you
should do the same :-) Calling String#toLowerCase() without an
explicit Locale is a bug, if you rely on locale-insensitive behavior.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
eMail: uwe@thetaphi.de
Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Balchandra Vaidya <ba...@oracle.com>.
Hi Uwe,
I will update your comments in the jbs bug.
Does this failure reproducible with Oracle JDK
<http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html>?
I could not
reproduce it on Linux (OL6 u5) and Solaris 11 so far. I will
try it on a Mac later.
Thanks
Balchandra
On 08/18/14 01:48 PM, Uwe Schindler wrote:
> Hi Balchandra:
>
> I checked it out with Java 7 on FreeBSD:
>
> ==== SNIP ====
> [uschindler@lucene ~]$ cat Test.java
> import java.util.Locale;
>
> public class Test {
>
> public static void main(final String[] args) throws Exception {
> Locale.setDefault(new Locale("tr", ""));
> Process exec = Runtime.getRuntime().exec("ls");
> }
>
> }
>
> [uschindler@lucene ~]$ javac Test.java
>
> [uschindler@lucene ~]$ java -version
> openjdk version "1.7.0_60"
> OpenJDK Runtime Environment (build 1.7.0_60-b19)
> OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
> [uschindler@lucene ~]$ java Test
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:450)
> at java.lang.Runtime.exec(Runtime.java:347)
> at Test.main(Test.java:7)
> ==== SNIP ====
>
> On MacOSX:
>
> ==== SNIP ====
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/javac Test.java
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java -version
> java version "1.6.0_65"
> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java Test
>
> -> So it seems that Apple's JDK6 is immune. Unfortunately I don’t have a FreeBSD version available. I also have no Solaris or AIX one.
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java -version
> java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java Test
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:104)
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:450)
> at java.lang.Runtime.exec(Runtime.java:347)
> at Test.main(Test.java:7)
>
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java -version
> java version "1.8.0_11"
> Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)
> macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java Test
> Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:105)
> at java.lang.UNIXProcess$1.run(UNIXProcess.java:94)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:92)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
> at java.lang.Runtime.exec(Runtime.java:620)
> at java.lang.Runtime.exec(Runtime.java:450)
> at java.lang.Runtime.exec(Runtime.java:347)
> at Test.main(Test.java:7)
> ==== SNIP ====
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> Sent: Monday, August 18, 2014 1:02 PM
>> To: 'Balchandra Vaidya'
>> Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
>> Subject: RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
>>
>> Hi Balchandra,
>>
>> it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
>> + /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
>> openjdk version "1.7.0_60"
>> OpenJDK Runtime Environment (build 1.7.0_60-b19) OpenJDK 64-Bit Server
>> VM (build 24.60-b09, mixed mode)
>>
>> and MacOSX (JDK 7u65):
>> + /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java
>> + -XX:+UseCompressedOops -XX:+UseG1GC -version
>> java version "1.7.0_65"
>> Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-
>> Bit Server VM (build 24.65-b04, mixed mode)
>>
>> See the attached failure notices from the Jenkins Server.
>>
>> After lunch I will do a quick check on command line and 3-liner Java test; any
>> MacOSX or BSD machine should be fine.
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>> -----Original Message-----
>>> From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
>>> Sent: Monday, August 18, 2014 12:39 PM
>>> To: Uwe Schindler
>>> Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
>>> Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in
>>> Runtime#exec()
>>>
>>>
>>> Hi Uwe,
>>>
>>> I have added your comments to JDK-8047340. The bug has an original
>>> comment "REGRESSION. Last worked in version 7u60"- are you sure the
>>> bug existed in JDK 7?
>>>
>>> Thanks
>>> Balchandra
>>>
>>>
>>>
>>> On 08/17/14 10:08 PM, Uwe Schindler wrote:
>>>> Hi,
>>>>
>>>> it took some time to find out, why sometimes Lucene/Solr's tests
>>>> failed with a very strange error message when trying to fork a
>>>> process on MacOSX and Linux. The reason for this bug to appear is
>>>> our randomized testing framework: As you know, we execute all tests
>>>> in different settings like default charsets, default locales or
>>>> default timezones, to find bugs caused by settings applied to the
>>>> environment, that may affect the code in an incorrect way. For more
>>>> information, see this (already famous) blog entry:
>>>> http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and
>>>> .h
>>>> tml
>>>>
>>>> In our case the JDK issue was:
>>>> https://bugs.openjdk.java.net/browse/JDK-8047340
>>>>
>>>> The issue on our side:
>>>> https://issues.apache.org/jira/browse/SOLR-6387
>>>>
>>>> We just want to add, that this bug also applies to Java 7 - the same
>>>> issue
>>> exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH
>>> /
>>> Locale.ROOT) should be backported to Java 7 and Java 6 for customers
>>> with support contract, too. As I cannot modify the bug report, it
>>> would be good to add issues for Java 7 and Java 6, too.
>>>> Uwe
>>>>
>>>> PS: We use the Forbidden-APIs checker
>>> (https://code.google.com/p/forbidden-apis/) to detect such bugs in
>>> Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other
>>> projects I don't know of yet. Maybe when building the rt.jar, you
>>> should do the same :-) Calling String#toLowerCase() without an
>>> explicit Locale is a bug, if you rely on locale-insensitive behavior.
>>>> -----
>>>> Uwe Schindler
>>>> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
>>>> eMail: uwe@thetaphi.de
>>>>
>>>>
>>>>
RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Balchandra:
I checked it out with Java 7 on FreeBSD:
==== SNIP ====
[uschindler@lucene ~]$ cat Test.java
import java.util.Locale;
public class Test {
public static void main(final String[] args) throws Exception {
Locale.setDefault(new Locale("tr", ""));
Process exec = Runtime.getRuntime().exec("ls");
}
}
[uschindler@lucene ~]$ javac Test.java
[uschindler@lucene ~]$ java -version
openjdk version "1.7.0_60"
OpenJDK Runtime Environment (build 1.7.0_60-b19)
OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
[uschindler@lucene ~]$ java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
==== SNIP ====
On MacOSX:
==== SNIP ====
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/javac Test.java
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java -version
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.6.0/bin/java Test
-> So it seems that Apple's JDK6 is immune. Unfortunately I don’t have a FreeBSD version available. I also have no Solaris or AIX one.
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java -version
java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.7.0/bin/java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:104)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java -version
java version "1.8.0_11"
Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)
macosx-vm:tmp jenkins$ ~/tools/java/64bit/jdk1.8.0/bin/java Test
Exception in thread "main" java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:105)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:94)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:92)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
at java.lang.Runtime.exec(Runtime.java:620)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at Test.main(Test.java:7)
==== SNIP ====
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Monday, August 18, 2014 1:02 PM
> To: 'Balchandra Vaidya'
> Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
> Subject: RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
>
> Hi Balchandra,
>
> it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
> + /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
> openjdk version "1.7.0_60"
> OpenJDK Runtime Environment (build 1.7.0_60-b19) OpenJDK 64-Bit Server
> VM (build 24.60-b09, mixed mode)
>
> and MacOSX (JDK 7u65):
> + /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java
> + -XX:+UseCompressedOops -XX:+UseG1GC -version
> java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-
> Bit Server VM (build 24.65-b04, mixed mode)
>
> See the attached failure notices from the Jenkins Server.
>
> After lunch I will do a quick check on command line and 3-liner Java test; any
> MacOSX or BSD machine should be fine.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
> > Sent: Monday, August 18, 2014 12:39 PM
> > To: Uwe Schindler
> > Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
> > Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in
> > Runtime#exec()
> >
> >
> > Hi Uwe,
> >
> > I have added your comments to JDK-8047340. The bug has an original
> > comment "REGRESSION. Last worked in version 7u60"- are you sure the
> > bug existed in JDK 7?
> >
> > Thanks
> > Balchandra
> >
> >
> >
> > On 08/17/14 10:08 PM, Uwe Schindler wrote:
> > > Hi,
> > >
> > > it took some time to find out, why sometimes Lucene/Solr's tests
> > > failed with a very strange error message when trying to fork a
> > > process on MacOSX and Linux. The reason for this bug to appear is
> > > our randomized testing framework: As you know, we execute all tests
> > > in different settings like default charsets, default locales or
> > > default timezones, to find bugs caused by settings applied to the
> > > environment, that may affect the code in an incorrect way. For more
> > > information, see this (already famous) blog entry:
> > > http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and
> > > .h
> > > tml
> > >
> > > In our case the JDK issue was:
> > > https://bugs.openjdk.java.net/browse/JDK-8047340
> > >
> > > The issue on our side:
> > > https://issues.apache.org/jira/browse/SOLR-6387
> > >
> > > We just want to add, that this bug also applies to Java 7 - the same
> > > issue
> > exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH
> > /
> > Locale.ROOT) should be backported to Java 7 and Java 6 for customers
> > with support contract, too. As I cannot modify the bug report, it
> > would be good to add issues for Java 7 and Java 6, too.
> > >
> > > Uwe
> > >
> > > PS: We use the Forbidden-APIs checker
> > (https://code.google.com/p/forbidden-apis/) to detect such bugs in
> > Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other
> > projects I don't know of yet. Maybe when building the rt.jar, you
> > should do the same :-) Calling String#toLowerCase() without an
> > explicit Locale is a bug, if you rely on locale-insensitive behavior.
> > >
> > > -----
> > > Uwe Schindler
> > > H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> > > eMail: uwe@thetaphi.de
> > >
> > >
> > >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Balchandra,
it definitely happened on Java 7 (FreeBSD port of OpenJDK, JDK 7u60):
+ /home/jenkins/tools/java/latest1.7/bin/java -XX:+UseG1GC -version
openjdk version "1.7.0_60"
OpenJDK Runtime Environment (build 1.7.0_60-b19)
OpenJDK 64-Bit Server VM (build 24.60-b09, mixed mode)
and MacOSX (JDK 7u65):
+ /Users/jenkins/tools/java/64bit/jdk1.7.0/bin/java -XX:+UseCompressedOops -XX:+UseG1GC -version
java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
See the attached failure notices from the Jenkins Server.
After lunch I will do a quick check on command line and 3-liner Java test; any MacOSX or BSD machine should be fine.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Balchandra Vaidya [mailto:balchandra.vaidya@oracle.com]
> Sent: Monday, August 18, 2014 12:39 PM
> To: Uwe Schindler
> Cc: rory.odonnell@oracle.com; 'Dalibor Topic'; dev@lucene.apache.org
> Subject: Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
>
>
> Hi Uwe,
>
> I have added your comments to JDK-8047340. The bug has an original
> comment "REGRESSION. Last worked in version 7u60"- are you sure the bug
> existed in JDK 7?
>
> Thanks
> Balchandra
>
>
>
> On 08/17/14 10:08 PM, Uwe Schindler wrote:
> > Hi,
> >
> > it took some time to find out, why sometimes Lucene/Solr's tests
> > failed with a very strange error message when trying to fork a process
> > on MacOSX and Linux. The reason for this bug to appear is our
> > randomized testing framework: As you know, we execute all tests in
> > different settings like default charsets, default locales or default
> > timezones, to find bugs caused by settings applied to the environment,
> > that may affect the code in an incorrect way. For more information,
> > see this (already famous) blog entry:
> > http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.h
> > tml
> >
> > In our case the JDK issue was:
> > https://bugs.openjdk.java.net/browse/JDK-8047340
> >
> > The issue on our side:
> > https://issues.apache.org/jira/browse/SOLR-6387
> >
> > We just want to add, that this bug also applies to Java 7 - the same issue
> exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH /
> Locale.ROOT) should be backported to Java 7 and Java 6 for customers with
> support contract, too. As I cannot modify the bug report, it would be good to
> add issues for Java 7 and Java 6, too.
> >
> > Uwe
> >
> > PS: We use the Forbidden-APIs checker
> (https://code.google.com/p/forbidden-apis/) to detect such bugs in Apache
> Lucene/Solr, Elasticsearch, Apache TIKA, and many other projects I don't
> know of yet. Maybe when building the rt.jar, you should do the same :-)
> Calling String#toLowerCase() without an explicit Locale is a bug, if you rely on
> locale-insensitive behavior.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >
Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by Balchandra Vaidya <ba...@oracle.com>.
Hi Uwe,
I have added your comments to JDK-8047340. The bug has an
original comment "REGRESSION. Last worked in version 7u60"- are
you sure the bug existed in JDK 7?
Thanks
Balchandra
On 08/17/14 10:08 PM, Uwe Schindler wrote:
> Hi,
>
> it took some time to find out, why sometimes Lucene/Solr's tests failed with a very strange error message when trying to fork a process on MacOSX and Linux. The reason for this bug to appear is our randomized testing framework: As you know, we execute all tests in different settings like default charsets, default locales or default timezones, to find bugs caused by settings applied to the environment, that may affect the code in an incorrect way. For more information, see this (already famous) blog entry: http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html
>
> In our case the JDK issue was:
> https://bugs.openjdk.java.net/browse/JDK-8047340
>
> The issue on our side:
> https://issues.apache.org/jira/browse/SOLR-6387
>
> We just want to add, that this bug also applies to Java 7 - the same issue exists there (and possibly also on Java 6). So the fix (Locale.ENGLISH / Locale.ROOT) should be backported to Java 7 and Java 6 for customers with support contract, too. As I cannot modify the bug report, it would be good to add issues for Java 7 and Java 6, too.
>
> Uwe
>
> PS: We use the Forbidden-APIs checker (https://code.google.com/p/forbidden-apis/) to detect such bugs in Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other projects I don't know of yet. Maybe when building the rt.jar, you should do the same :-) Calling String#toLowerCase() without an explicit Locale is a bug, if you rely on locale-insensitive behavior.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Apache Lucene/Solr: Turkish lowercasing bug in Runtime#exec()
Posted by dalibor topic <da...@oracle.com>.
On 17.08.2014 23:08, Uwe Schindler wrote:
> PS: We use the Forbidden-APIs checker (https://code.google.com/p/forbidden-apis/) to detect such bugs in Apache Lucene/Solr, Elasticsearch, Apache TIKA, and many other projects I don't know of yet. Maybe when building the rt.jar, you should do the same :-) Calling String#toLowerCase() without an explicit Locale is a bug, if you rely on locale-insensitive behavior.
I think that's a fine idea to discuss on jdk9-dev, if you want to start
a separate thread there. There is a current JEP draft focused on
cleaning up the sources wrt to lint/doclint cleanness:
http://openjdk.java.net/jeps/8042878
and the background for that JEP is explained in this thread:
http://mail.openjdk.java.net/pipermail/jdk9-dev/2013-December/000141.html
There is other ongoing cleanup-focused work targeting low hanging fruit
in JDK sources led by individuals in the OpenJDK community [0]. There
could always be more ;)
cheers,
dalibor topic
[0] http://hg.openjdk.java.net/jdk9/jdk9/jdk/log?rev=otavio
--
<http://www.oracle.com> Dalibor Topic | Principal Product Manager
Phone: +494089091214 <tel:+494089091214> | Mobile: +491737185961
<tel:+491737185961>
ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg
ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603
Geschäftsführer: Jürgen Kunz
Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher
<http://www.oracle.com/commitment> Oracle is committed to developing
practices and products that help protect the environment
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org