You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/12/09 14:53:01 UTC
[jira] Created: (SOLR-2279) Add a MockDirectoryFactory (or similar)
for Solr tests
Add a MockDirectoryFactory (or similar) for Solr tests
------------------------------------------------------
Key: SOLR-2279
URL: https://issues.apache.org/jira/browse/SOLR-2279
Project: Solr
Issue Type: Test
Components: Build
Reporter: Robert Muir
Fix For: 3.1, 4.0
Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
This has a number of advantages:
* By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
* The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
first opened the directory.
* MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
* MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
* MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042103#comment-13042103 ]
Shai Erera commented on SOLR-2279:
----------------------------------
I debug-traced MultiCoreEmbeddedTest and this is what I found: the test opens two indexes, one under a temp directory, and one under trunk/solr/example/multicore/core0/data/index. The one under temp is populated alright (and exists), while the one under solr/example (to clarify -- this directory is expected to be found on the checkout, so it seems) is empty, and hence the test fails on IndexNotFoundException.
I don't understand the test, so I cannot fix it. Just put it here in case someone else knows this code. I don't understand why this test passes w/ RAMDirectoryFactory.
Also, under solr/example/multicore/core0/conf, solrconfig.xml lists dirFactory to be StandardDirFactory. I don't know what are the implications of this on the test, but pointing it out as well.
Hope this info helps someone debug and fix the test :).
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-2279:
------------------------------
Attachment: SOLR-2279.patch
attached is a committable patch, changes from previous:
* added to uima contrib too
* disabled for the two problematic tests.
I think its ok for now that we disable this factory for two tests like such in setUp():
{noformat}
// TODO: fix this test to use MockDirectoryFactory
System.clearProperty("solr.directoryFactory");
{noformat}
Hopefully we can fix these tests in the future, but for now this is a good improvement in test coverage. I'll test on windows now, and commit this as a first step if everything is ok. I'll keep the issue open because I think we want to fix those 2 tests.
Also, I was surprised to find no problems with the spellchecker, but the reason for this is that it doesnt use the DirectoryFactory in solr, instead just FSDirectory.open! (I wonder if this should be improved separately?)
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2279) Add a MockDirectoryFactory (or similar)
for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-2279:
------------------------------
Fix Version/s: (was: 3.1)
moving out.. i don't see myself fixing this test issue very quickly.
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042052#comment-13042052 ]
Robert Muir commented on SOLR-2279:
-----------------------------------
Ok, so now i can explain why MultiCoreEmbeddedTest only failed on trunk,
its because we System.clearProperty()'ed in a previous test to disable MDW,
which disabled it for all subsequent tests in the JVM.
I'm fixing this now so its reset in beforeClass().
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved SOLR-2279.
-------------------------------
Resolution: Fixed
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir reassigned SOLR-2279:
---------------------------------
Assignee: Robert Muir
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042057#comment-13042057 ]
Robert Muir commented on SOLR-2279:
-----------------------------------
Now that all 3 tests fail consistently with MockDirectoryWrapper, I suspect its
something in these base multicore test classes...
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042041#comment-13042041 ]
Robert Muir commented on SOLR-2279:
-----------------------------------
I committed the initial patch in 1130042, and backported to branch3x in 1130044.
So the remaining tests to figure out are the ones with the TODO:
* MergeIndexesEmbeddedTest (trunk, branch_3x)
* MultiCoreExampleJettyTest (trunk, branch_3x)
* MultiCoreEmbeddedTest <-- strangely enough, this one only on branch_3x
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-2279:
------------------------------
Attachment: SOLR-2279.patch
updated patch: I really want to be able to track file handles in Solr's tests, so I added a hack to avoid the issue of Directory.close() never being called.
A couple tests fail using MockDirectoryWrapperFactory, still trying to track down why this is.
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or
similar) for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-2279:
------------------------------
Fix Version/s: 3.3
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2279) Add a MockDirectoryFactory (or similar)
for Solr tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-2279:
------------------------------
Attachment: SOLR-2279.patch
here's a patch, but many solr tests fail because as Shai mentioned, they don't close() their Directory instances.
I think this is a pretty serious bug. While it may be the case that this works with our supplied directories, i think for "DirectoryFactory" to be exposed as a public extension point, we should call .close().
> Add a MockDirectoryFactory (or similar) for Solr tests
> ------------------------------------------------------
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and soon-to-be added newFSDirectory() which always ensures the directory returned is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing a change to one of our directory implementations, we can run all tests with it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these directories are properly closed, if not, it causes the test to fail with a stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks by default, when you open a file they save the stacktrace of where you opened it from. If you try to close the directory without say, closing an IndexReader, it fails with the stacktrace of where you opened the reader from. This is helpful for tracking down resource leaks. Currently Solr warns if it cannot delete its test temporary directory, but this is better since you know exactly where the resource leak came from. This can be disabled with an optional setter which we should probably expose for some tests that have known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as simulating a crash, simulating disk full, emulating windows (where you can't delete open files), etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org