You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Jason Gerlowski <ge...@gmail.com> on 2014/11/12 16:06:36 UTC
LuceneTestCase static method usage
Hi all,
I'm seeing NPE's when calling static methods on LuceneTestCase (without
extending it).
I'm trying to write tests for a few classes that interact with Lucene. To
do that, I was trying to create JUnit @Rule (
https://github.com/junit-team/junit/wiki/Rules) fixtures that I can share
across different test classes. I want these fixtures to use the
randomness/extra-checks found in LuceneTestCase, but I can't extend LTC
because my fixtures already need to extend JUnit's 'Rule' class for JUnit
to know how to use them.
So instead I wrote the fixtures to access LTC functionality though the
class's static methods (LTC.newDirectory(), LTC.newSearcher(), etc.)
Is this a valid way to access LTC methods? Is there any special
initialization I need to do before using the class in this way?
I ask because I've started to see a few occasional NPE's in LTC during
tests that use these fixtures. To an amateur, it looks like some static
fields in LTC aren't being initialized. It's hard to tell whether I should
consider this a bug in the class, or whether I'm using it incorrectly.
Thanks for any help/insight you can offer!
Best,
Jason Gerlowski
(I can follow-up with code-snippets that reproduce this issue. I didn't
post them in this email because I thought I might just be misusing
LuceneTestCase).
Re: LuceneTestCase static method usage
Posted by Chris Hostetter <ho...@fucit.org>.
Posting the specific details of the NPEs you are seeing (ie: stack
traces) would help to answer your question.
Some of the functionality in LTC is definitely tied to the JUnit test case
lifecycle, so it's certainly possible that some static objects used by
static methods aren't being initialized yet when you make your calls from
rules -- but ewther that's a bug or a neccessity (ie: maybe can'd
do "newSearcher" until the the randomseed is set and that's set y
another Rule?) remains to be seen.
: Date: Wed, 12 Nov 2014 10:06:36 -0500
: From: Jason Gerlowski <ge...@gmail.com>
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: LuceneTestCase static method usage
:
: Hi all,
:
: I'm seeing NPE's when calling static methods on LuceneTestCase (without
: extending it).
:
: I'm trying to write tests for a few classes that interact with Lucene. To
: do that, I was trying to create JUnit @Rule (
: https://github.com/junit-team/junit/wiki/Rules) fixtures that I can share
: across different test classes. I want these fixtures to use the
: randomness/extra-checks found in LuceneTestCase, but I can't extend LTC
: because my fixtures already need to extend JUnit's 'Rule' class for JUnit
: to know how to use them.
:
: So instead I wrote the fixtures to access LTC functionality though the
: class's static methods (LTC.newDirectory(), LTC.newSearcher(), etc.)
:
: Is this a valid way to access LTC methods? Is there any special
: initialization I need to do before using the class in this way?
:
: I ask because I've started to see a few occasional NPE's in LTC during
: tests that use these fixtures. To an amateur, it looks like some static
: fields in LTC aren't being initialized. It's hard to tell whether I should
: consider this a bug in the class, or whether I'm using it incorrectly.
:
: Thanks for any help/insight you can offer!
:
: Best,
:
: Jason Gerlowski
:
:
: (I can follow-up with code-snippets that reproduce this issue. I didn't
: post them in this email because I thought I might just be misusing
: LuceneTestCase).
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase static method usage
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Ah, yes -- you won't be able to do this.The reason is LuceneTestCase
itself has rules that initialize its own data structures. Some of LTC
methods rely only on the randomized context, but some do rely on prior
initialization. This particular one relies on:
/** Set by TestRuleSetupAndRestoreClassEnv */
static LiveIWCFlushMode liveIWCFlushMode;
I'm afraid there isn't an easy solution to your problem. Or rather
there is: your suite needs to extend the LTC if you wish to reuse its
infrastructure. There are a lot more benefits of extending LTC than
just utility methods -- a number of configuration items sit as
annotations on this class. It's worth it.
Any other workaround will be a dirty hack. Highly discouraged.
...One of such workarounds would be to create a test suite like (pseudocode):
public class MyForkingSuite extends LuceneTestCase {
public void testMyStuff() {
// you're within a test method, everything is initialized. Simply
fork a *regular* JUnit runner.
Result r = JUnit.run(MyClass.class);
assert r is successfull;
}
}
This is super dirty... but will work as long as your MyClass doesn't
try to create a nested RandomizedRunner (these can't be nested).
Dawid
On Thu, Nov 13, 2014 at 4:22 AM, Jason Gerlowski <ge...@gmail.com> wrote:
> To add some additional information, the stack trace I'm seeing is.:
>
> java.lang.NullPointerException
> at
> org.apache.lucene.util.LuceneTestCase.maybeChangeLiveIndexWriterConfig(LuceneTestCase.java:1080)
> at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:103)
> at
> org.apache.lucene.index.LuceneIndexer.indexOneDocument(LuceneIndexer.java:55)
> at
> org.apache.lucene.index.TestSample.indexAnimalTestData(TestSample.java:45)
> at org.apache.lucene.index.TestSample.setUp(TestSample.java:24)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:401)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:642)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:129)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:559)
>
>
> This is the indexing @Rule fixture I've tried putting together:
>
> public class LuceneIndexer extends ExternalResource {
> public static final String TEST_FIELD_NAME = "TEST_FIELD_NAME";
>
> private final Analyzer analyzer;
> private final Directory directory;
> private RandomIndexWriter writer;
>
> public LuceneIndexer(Directory directory, Analyzer analyzer) {
> this.directory = directory;
> this.analyzer = analyzer;
> }
>
> public Directory getDirectory() {
> return directory;
> }
>
> @Override
> public void before() throws IOException {
> writer = new RandomIndexWriter(LuceneTestCase.random(), directory,
> new IndexWriterConfig(analyzer));
> }
>
> @Override
> public void after() {
> try {
> IOUtils.close(directory, analyzer);
> } catch (IOException e) {
> throw new RuntimeException(e);
> }
> }
>
> public void close() throws IOException {
> IOUtils.close(writer);
> }
>
> public void indexOneDocument(String text) throws IOException {
> Document document = createDocumentFromText(text);
> writer.addDocument(document);
> }
>
> private Document createDocumentFromText(String documentText) {
> Document document = new Document();
> FieldType defaultFieldConfig = new FieldType();
>
> defaultFieldConfig.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
> Field field = new Field(TEST_FIELD_NAME, documentText,
> defaultFieldConfig);
> document.add(field);
>
> return document;
> }
> }
>
> I'm reproducing the issue with the following test case below:
>
> @RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class)
> public class TestSample {
>
> @Rule
> public LuceneIndexer indexerFixture = new
> LuceneIndexer(LuceneTestCase.newDirectory(), new
> MockAnalyzer(LuceneTestCase.random()));
>
> @Before
> public void setUp() throws IOException {
> indexerFixture.indexOneDocument("document");
> indexerFixture.indexOneDocument("another document");
> indexerFixture.indexOneDocument("third document");
> // NPE occurs when this last call gets down to
> LuceneTestCase.maybeChangeLiveIndexWriterConfig()
> indexerFixture.indexOneDocument("last document");
>
> indexerFixture.close();
> }
>
> @Test
> public void first_reader() {
> System.out.println("In first test");
> }
>
> @Test
> public void second_test() {
> System.out.println("In second test");
> }
> }
>
>
> On Wed, Nov 12, 2014 at 2:51 PM, Dawid Weiss <da...@cs.put.poznan.pl>
> wrote:
>>
>> > it looks like some static fields in LTC aren't being initialized.
>>
>> If your code is executed within a static initializer then these fields
>> won't be initialized. Like Chris said -- post the full stack trace
>> and, ideally, a short snippet of code demonstrating what you're doing
>> (or trying to do).
>>
>> > I can't extend LTC because my fixtures already need to extend JUnit's
>> > 'Rule' class for JUnit to know how to use them
>>
>> This isn't true. Your suite class can extend LTC and you can add
>> additional rules within that class. You probably mean something else
>> -- be specific, provide code examples.
>>
>> Dawid
>>
>> On Wed, Nov 12, 2014 at 4:06 PM, Jason Gerlowski <ge...@gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > I'm seeing NPE's when calling static methods on LuceneTestCase (without
>> > extending it).
>> >
>> > I'm trying to write tests for a few classes that interact with Lucene.
>> > To
>> > do that, I was trying to create JUnit @Rule
>> > (https://github.com/junit-team/junit/wiki/Rules) fixtures that I can
>> > share
>> > across different test classes. I want these fixtures to use the
>> > randomness/extra-checks found in LuceneTestCase, but I can't extend LTC
>> > because my fixtures already need to extend JUnit's 'Rule' class for
>> > JUnit to
>> > know how to use them.
>> >
>> > So instead I wrote the fixtures to access LTC functionality though the
>> > class's static methods (LTC.newDirectory(), LTC.newSearcher(), etc.)
>> >
>> > Is this a valid way to access LTC methods? Is there any special
>> > initialization I need to do before using the class in this way?
>> >
>> > I ask because I've started to see a few occasional NPE's in LTC during
>> > tests
>> > that use these fixtures. To an amateur, it looks like some static
>> > fields in
>> > LTC aren't being initialized. It's hard to tell whether I should
>> > consider
>> > this a bug in the class, or whether I'm using it incorrectly.
>> >
>> > Thanks for any help/insight you can offer!
>> >
>> > Best,
>> >
>> > Jason Gerlowski
>> >
>> >
>> > (I can follow-up with code-snippets that reproduce this issue. I didn't
>> > post them in this email because I thought I might just be misusing
>> > LuceneTestCase).
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase static method usage
Posted by Jason Gerlowski <ge...@gmail.com>.
To add some additional information, the stack trace I'm seeing is.:
java.lang.NullPointerException
at
org.apache.lucene.util.LuceneTestCase.maybeChangeLiveIndexWriterConfig(LuceneTestCase.java:1080)
at
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:103)
at
org.apache.lucene.index.LuceneIndexer.indexOneDocument(LuceneIndexer.java:55)
at
org.apache.lucene.index.TestSample.indexAnimalTestData(TestSample.java:45)
at org.apache.lucene.index.TestSample.setUp(TestSample.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:401)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:642)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:129)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:559)
This is the indexing @Rule fixture I've tried putting together:
public class LuceneIndexer extends ExternalResource {
public static final String TEST_FIELD_NAME = "TEST_FIELD_NAME";
private final Analyzer analyzer;
private final Directory directory;
private RandomIndexWriter writer;
public LuceneIndexer(Directory directory, Analyzer analyzer) {
this.directory = directory;
this.analyzer = analyzer;
}
public Directory getDirectory() {
return directory;
}
@Override
public void before() throws IOException {
writer = new RandomIndexWriter(LuceneTestCase.random(), directory,
new IndexWriterConfig(analyzer));
}
@Override
public void after() {
try {
IOUtils.close(directory, analyzer);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public void close() throws IOException {
IOUtils.close(writer);
}
public void indexOneDocument(String text) throws IOException {
Document document = createDocumentFromText(text);
writer.addDocument(document);
}
private Document createDocumentFromText(String documentText) {
Document document = new Document();
FieldType defaultFieldConfig = new FieldType();
defaultFieldConfig.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
Field field = new Field(TEST_FIELD_NAME, documentText,
defaultFieldConfig);
document.add(field);
return document;
}
}
I'm reproducing the issue with the following test case below:
@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class)
public class TestSample {
@Rule
public LuceneIndexer indexerFixture = new
LuceneIndexer(LuceneTestCase.newDirectory(), new
MockAnalyzer(LuceneTestCase.random()));
@Before
public void setUp() throws IOException {
indexerFixture.indexOneDocument("document");
indexerFixture.indexOneDocument("another document");
indexerFixture.indexOneDocument("third document");
// NPE occurs when this last call gets down to
LuceneTestCase.maybeChangeLiveIndexWriterConfig()
indexerFixture.indexOneDocument("last document");
indexerFixture.close();
}
@Test
public void first_reader() {
System.out.println("In first test");
}
@Test
public void second_test() {
System.out.println("In second test");
}
}
On Wed, Nov 12, 2014 at 2:51 PM, Dawid Weiss <da...@cs.put.poznan.pl>
wrote:
> > it looks like some static fields in LTC aren't being initialized.
>
> If your code is executed within a static initializer then these fields
> won't be initialized. Like Chris said -- post the full stack trace
> and, ideally, a short snippet of code demonstrating what you're doing
> (or trying to do).
>
> > I can't extend LTC because my fixtures already need to extend JUnit's
> 'Rule' class for JUnit to know how to use them
>
> This isn't true. Your suite class can extend LTC and you can add
> additional rules within that class. You probably mean something else
> -- be specific, provide code examples.
>
> Dawid
>
> On Wed, Nov 12, 2014 at 4:06 PM, Jason Gerlowski <ge...@gmail.com>
> wrote:
> > Hi all,
> >
> > I'm seeing NPE's when calling static methods on LuceneTestCase (without
> > extending it).
> >
> > I'm trying to write tests for a few classes that interact with Lucene.
> To
> > do that, I was trying to create JUnit @Rule
> > (https://github.com/junit-team/junit/wiki/Rules) fixtures that I can
> share
> > across different test classes. I want these fixtures to use the
> > randomness/extra-checks found in LuceneTestCase, but I can't extend LTC
> > because my fixtures already need to extend JUnit's 'Rule' class for
> JUnit to
> > know how to use them.
> >
> > So instead I wrote the fixtures to access LTC functionality though the
> > class's static methods (LTC.newDirectory(), LTC.newSearcher(), etc.)
> >
> > Is this a valid way to access LTC methods? Is there any special
> > initialization I need to do before using the class in this way?
> >
> > I ask because I've started to see a few occasional NPE's in LTC during
> tests
> > that use these fixtures. To an amateur, it looks like some static
> fields in
> > LTC aren't being initialized. It's hard to tell whether I should
> consider
> > this a bug in the class, or whether I'm using it incorrectly.
> >
> > Thanks for any help/insight you can offer!
> >
> > Best,
> >
> > Jason Gerlowski
> >
> >
> > (I can follow-up with code-snippets that reproduce this issue. I didn't
> > post them in this email because I thought I might just be misusing
> > LuceneTestCase).
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: LuceneTestCase static method usage
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> it looks like some static fields in LTC aren't being initialized.
If your code is executed within a static initializer then these fields
won't be initialized. Like Chris said -- post the full stack trace
and, ideally, a short snippet of code demonstrating what you're doing
(or trying to do).
> I can't extend LTC because my fixtures already need to extend JUnit's 'Rule' class for JUnit to know how to use them
This isn't true. Your suite class can extend LTC and you can add
additional rules within that class. You probably mean something else
-- be specific, provide code examples.
Dawid
On Wed, Nov 12, 2014 at 4:06 PM, Jason Gerlowski <ge...@gmail.com> wrote:
> Hi all,
>
> I'm seeing NPE's when calling static methods on LuceneTestCase (without
> extending it).
>
> I'm trying to write tests for a few classes that interact with Lucene. To
> do that, I was trying to create JUnit @Rule
> (https://github.com/junit-team/junit/wiki/Rules) fixtures that I can share
> across different test classes. I want these fixtures to use the
> randomness/extra-checks found in LuceneTestCase, but I can't extend LTC
> because my fixtures already need to extend JUnit's 'Rule' class for JUnit to
> know how to use them.
>
> So instead I wrote the fixtures to access LTC functionality though the
> class's static methods (LTC.newDirectory(), LTC.newSearcher(), etc.)
>
> Is this a valid way to access LTC methods? Is there any special
> initialization I need to do before using the class in this way?
>
> I ask because I've started to see a few occasional NPE's in LTC during tests
> that use these fixtures. To an amateur, it looks like some static fields in
> LTC aren't being initialized. It's hard to tell whether I should consider
> this a bug in the class, or whether I'm using it incorrectly.
>
> Thanks for any help/insight you can offer!
>
> Best,
>
> Jason Gerlowski
>
>
> (I can follow-up with code-snippets that reproduce this issue. I didn't
> post them in this email because I thought I might just be misusing
> LuceneTestCase).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org