You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Gus Heck (Jira)" <ji...@apache.org> on 2020/02/18 19:42:00 UTC
[jira] [Commented] (SOLR-14217) tests respect tests.workDir
correctly (prevent SSD destruction)
[ https://issues.apache.org/jira/browse/SOLR-14217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039400#comment-17039400 ]
Gus Heck commented on SOLR-14217:
---------------------------------
I think we at least need a null check here... but you probably need to also account for the null downstream, since I expect you added this property to use it somewhere
{code:java}
// set solr.install.dir needed by some test configs outside of the test sandbox (!)
System.setProperty("solr.install.dir", ExternalPaths.SOURCE_HOME);
{code}
This causes failures for folks using the solr test infrastructure in projects other than solr because SOURCE_HOME can be null. That leads to:
{code:java}
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
at java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
at java.base/java.util.Properties.put(Properties.java:1337)
at java.base/java.util.Properties.setProperty(Properties.java:225)
at java.base/java.lang.System.setProperty(System.java:895)
at org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
> tests respect tests.workDir correctly (prevent SSD destruction)
> ---------------------------------------------------------------
>
> Key: SOLR-14217
> URL: https://issues.apache.org/jira/browse/SOLR-14217
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
> Reporter: Robert Muir
> Priority: Major
> Fix For: 8.5
>
> Attachments: SOLR-14217.patch
>
>
> To prevent SSD destruction you should be able to set in your {{gradle.properties}}:
> {code}
> tests.workDir=/tmp/lucene_gradle
> {code}
> This causes all tests to write their temporary stuff (e.g. indexes) to that location, instead of build/ directories inside the checkout.
> Unfortunately if you do this, some solr tests fail, because their configs will then try to reach out to invalid places. It happens because they load up configuration files with logic that looks like this:
> {code}
> <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
> {code}
> Because {{solr.install.dir}} is not set, it falls back to an inappropriate relative path, and tries to access bogus locations on the filesystem, causing tests to fail:
> {noformat}
> 2> Caused by: org.apache.solr.common.SolrException: Error loading solr config from /tmp/lucene_gradle/solr.client.solrj.embedded.SolrExampleJettyTest_FF93943C1C04FCAC-001/tempDir-001/collection1/conf/solrconfig.xml
> 2> at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:157)
> 2> at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:111)
> 2> at org.apache.solr.core.ConfigSetService.loadConfigSet(ConfigSetService.java:82)
> 2> ... 8 more
> 2> Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/tmp/lucene_gradle/solr.client.solrj.embedded.SolrExampleJettyTest_FF93943C1C04FCAC-001/tempDir-001/collection1/../../../../contrib/extraction/lib" "read")
> 2> at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> 2> at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
> 2> at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
> 2> at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:747)
> 2> at java.base/sun.nio.fs.UnixPath.checkRead(UnixPath.java:818)
> 2> at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:399)
> 2> at java.base/java.nio.file.Files.newDirectoryStream(Files.java:607)
> 2> at org.apache.solr.core.SolrResourceLoader.getURLs(SolrResourceLoader.java:286)
> 2> at org.apache.solr.core.SolrResourceLoader.getFilteredURLs(SolrResourceLoader.java:319)
> 2> at org.apache.solr.core.SolrConfig.initLibs(SolrConfig.java:782)
> 2> at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:176)
> 2> at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:149)
> 2> ... 10 more
> {noformat}
> The system property must be set so that these tests will work with the configured tests.workDir
> It has no impact on tests speed, because solr tests are all sitting in Thread.sleep, not bottlenecked on anything related to this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org