You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (Created) (JIRA)" <ji...@apache.org> on 2012/03/27 08:59:32 UTC
[jira] [Created] (LUCENE-3927) allow running trunk tests with IBM
JRE
allow running trunk tests with IBM JRE
--------------------------------------
Key: LUCENE-3927
URL: https://issues.apache.org/jira/browse/LUCENE-3927
Project: Lucene - Java
Issue Type: Task
Components: general/test
Affects Versions: 4.0
Reporter: Robert Muir
This is currently not possible because of how the SPI loader works,
we cannot simulate Lucene3x codec with PreFlexRWCodec.
But we should still allow basic testing (even though we cannot test preflex).
After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239303#comment-13239303 ]
Robert Muir commented on LUCENE-3927:
-------------------------------------
The hack in the patch is not hardcoded at all IBM: it checks if the impersonation actually works too.
{code}
Constants.JAVA_VENDOR.startsWith("IBM") && (!(Codec.forName("Lucene3x") instanceof PreFlexRWCodec));
{code}
For other JREs we still throw an assertion error saying to fix your classpath.
But again, we don't necessarily need to commit this patch. Again more interesting is the fact that,
with the patch, if i run tests I get 'too many open files' from TestShardSearching, and I get test
failures from Solr's CSV support...
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Shai Erera (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239287#comment-13239287 ]
Shai Erera commented on LUCENE-3927:
------------------------------------
I reported a bug report on that and it was fixed. I cannot locate now my report (the link I have comes up empty) nor in which version they fixed it. But I have at hand a fixed (unofficial) jar and tests pass with it (without Robert's patch). I will try to find the version which contains the fix.
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Dawid Weiss (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239277#comment-13239277 ]
Dawid Weiss commented on LUCENE-3927:
-------------------------------------
bq. t uses a HashMap instead of LinkedHashMap as lookup cache and that mixes the SPI classes up.
This is sad. It's a simple bug to fix but will never be probably...
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Shai Erera (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239300#comment-13239300 ]
Shai Erera commented on LUCENE-3927:
------------------------------------
I agree. But if we know in which version it has been fixed, we can at least put a comment to remove that hack. Or if have an alternative solution that does not require that fix, it's better.
Because as I understand, that patch disables PreFlex for all IBM JREs right?
Anyway, feel free to proceed with that patch because it at least enables running trunk tests with an IBM JRE.
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239327#comment-13239327 ]
Robert Muir commented on LUCENE-3927:
-------------------------------------
{code}
if (!(Codec.forName("Lucene3x") instanceof PreFlexRWCodec)) {
System.err.println("ERROR: Your VM's java.util.ServiceLoader implementation is buggy"+
" and does not respect classpath order, please report this to the vendor.");
{code}
I think this could be confusing to people using Sun JVMs, but just with the wrong
classpath? There is currently an assert for this:
{code}
assert (codec instanceof PreFlexRWCodec) : "fix your classpath to have tests-framework.jar before lucene-core.jar";
{code}
So I think we should just be careful about the warning to not confuse (maybe check Constants.JRE_VENDOR to
make sure we send the correct message?)
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch, LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239298#comment-13239298 ]
Robert Muir commented on LUCENE-3927:
-------------------------------------
Well mainly I wanted to get past this (at least have a patch available!) because i see interesting
test failures in trunk with the IBM JVM...
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3927:
--------------------------------
Attachment: LUCENE-3927.patch
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Shai Erera (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239296#comment-13239296 ]
Shai Erera commented on LUCENE-3927:
------------------------------------
Still can't find it. I tried with the latest JRE "IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Windows 7 amd64-64 jvmwa6460sr10-20111207_96808 (JIT enabled, AOT enabled)" but TestImpersonation fails. So I guess the fix is not yet included in that version.
I know it has been fixed (because as I said, I have an unofficial JAR with which I can run Trunk tests), so I will try newer releases when they're available.
But if you can resolve it such that it works without that fix, that will be great.
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3927:
----------------------------------
Attachment: LUCENE-3927.patch
Here is a patch with a workaround for the JRE bug. It forcefully registers the preflex rw codec by reflection and prints warning (once per JVM).
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch, LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-3927.
---------------------------------
Resolution: Fixed
Fix Version/s: 4.0
Thanks Uwe!
I really like this solution: if people have screwed up classpaths they get a warning. If they have a buggy IBM JRE they get a warning. but 'ant test' works.
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3927.patch, LUCENE-3927.patch, LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3927:
--------------------------------
Attachment: LUCENE-3927.patch
Uwe's patch but with the wording tweaked for non-IBM JRE :)
I think this is ready to commit. Thanks Uwe!
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch, LUCENE-3927.patch, LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239274#comment-13239274 ]
Robert Muir commented on LUCENE-3927:
-------------------------------------
That would be a better solution than my hack (which swaps in SimpleText as punishment).
Though such an override is pretty dangerous, anyone messing with this is likely to just not
be able to read their index. Maybe we can make it private/package-private and setAccessible?
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3927) allow running trunk tests with IBM
JRE
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239271#comment-13239271 ]
Uwe Schindler commented on LUCENE-3927:
---------------------------------------
This bug in IBM JDK goes back to Apache Harmony, where you can see how the SPI code is implemented. It uses a HashMap instead of LinkedHashMap as lookup cache and that mixes the SPI classes up. This is a bug, but unfortunately the API is not clear about that.
Maybe we add a "override" option to SPILoader, so you can force NamedSPILoader to return a specific impl, ignoring the classpath? By this we can override the "Lucene3x" codec to be "new PreflexRWCodec()" programmatically.
> allow running trunk tests with IBM JRE
> --------------------------------------
>
> Key: LUCENE-3927
> URL: https://issues.apache.org/jira/browse/LUCENE-3927
> Project: Lucene - Java
> Issue Type: Task
> Components: general/test
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3927.patch
>
>
> This is currently not possible because of how the SPI loader works,
> we cannot simulate Lucene3x codec with PreFlexRWCodec.
> But we should still allow basic testing (even though we cannot test preflex).
> After hacking around the issue, I get interesting fails with this JRE so I think its worth it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org