You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Joe Pollard <Jo...@bazaarvoice.com> on 2009/03/27 16:59:42 UTC
Best way to unit test solr integration
Hello,
On our project, we have quite a bit of code used to generate Solr queries, and I need to create some unit tests to ensure that these continue to work. In addition, I need to generate some unit tests that will test indexing and retrieval of certain documents, based on our current schema and the application logic that generates the indexable documents as well as generates the Solr queries.
My question is - what's the best way for me to unit test our Solr integration?
I'd like to be able to spin up an embedded/in-memory solr, or that failing just start one up as part of my test case setup, fill it with interesting documents, and do some queries, comparing the results to expected results.
Are there wiki pages or other documented examples of doing this? It seems rather straight-forward, but who knows, it may be dead simple with some unknown feature.
Thanks!
-Joe
Re: Best way to unit test solr integration
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Joe,
Have a look at Solr's own unit test, I believe they have pieces of what you need - the ability to start a Solr instance, index docs, run a query, and test if the results contain what you expect to see in them. You can get to Solr's unit test by checking out Solr from svn, or by browising the svn repository via the Web.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
> From: Joe Pollard <Jo...@bazaarvoice.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Friday, March 27, 2009 12:50:31 PM
> Subject: RE: Best way to unit test solr integration
>
> Thanks for the tips, I like the suggestion of testing the document and query
> generation without having solr involved. That seems like a more bite-sized
> unit; I think I'll do that.
>
> However, here's the test case that I'm considering where I'd like to have a live
> solr instance:
>
> During an exercise of optimizing our schema, I'm going to be making wholesale
> changes that I'd like to ensure don't break some portion of our app. It seems
> like a good method for this would be to write a test with the following steps:
> (arguably not a unit test, but a very valuable test indeed in our application)
> * take some defined model object generated at test time, store it in db
> * run it through our document creation code
> * submit it into solr
> * generate a query using our custom criteria-based generation code
> * ensure that the query returns the results as expected
> * flesh out the new model objects from the db using only the id fields returned
> from Solr
> * In the end, it would be expected to have model objects retrieved from the db
> that match model objects at the beginning of the test.
>
> These building blocks could be stacked in numerous ways to test almost all the
> different scenarios in which we use Solr.
>
> Also, when/if we start making solr config changes, I can ensure that they change
> nothing from my app's functional point of view (with the exception of ridding us
> of dreaded OOMs).
>
> Thanks,
> -Joe
>
> -----Original Message-----
> From: Eric Pugh [mailto:epugh@opensourceconnections.com]
> Sent: Friday, March 27, 2009 11:27 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Best way to unit test solr integration
>
> So my first thought is that "unit test + solr integration" is an
> oxymoron. In the sense that unit test implies the smallest functional
> unit, and solr integration implies multiple units working together.
>
> It sounds like you have two different tasks. the code that generate
> queies, you can test that without Solr. If you need to parse some
> sort of solr document to generate a query based on it, then mock up
> the query. A lot of folks will just use Solr to build a result set,
> and then save that on the filesystem. "my_big_result1.xml" and read
> it in and feed it to your code.
>
> On the other hand, for you code testing indexing and retrieval, again,
> if you can use the same approach to decouple what solr does from your
> code. Unless you've patched Solr, you shouldn't need to unit test
> Solr, Solr has very nice unit testing built in.
>
> On the other hand, if you are doing integration testing, where you
> want a more end to end view of your application, then you probably
> already have a "test" solr setup in your environment somewhere that
> you can rely on to use.
>
> Spinning up and shutting down Solr for tests can be done, and I can
> think of use cases for why you might want to do it, but it does incur
> a penalty of being more work. And you still need to validate that
> your "embedded/unit test" solr works the same as your integration/test
> environment Solr.
>
> Eric
>
>
>
> On Mar 27, 2009, at 11:59 AM, Joe Pollard wrote:
>
> > Hello,
> >
> > On our project, we have quite a bit of code used to generate Solr
> > queries, and I need to create some unit tests to ensure that these
> > continue to work. In addition, I need to generate some unit tests
> > that will test indexing and retrieval of certain documents, based on
> > our current schema and the application logic that generates the
> > indexable documents as well as generates the Solr queries.
> >
> > My question is - what's the best way for me to unit test our Solr
> > integration?
> >
> > I'd like to be able to spin up an embedded/in-memory solr, or that
> > failing just start one up as part of my test case setup, fill it
> > with interesting documents, and do some queries, comparing the
> > results to expected results.
> >
> > Are there wiki pages or other documented examples of doing this? It
> > seems rather straight-forward, but who knows, it may be dead simple
> > with some unknown feature.
> >
> > Thanks!
> > -Joe
>
> -----------------------------------------------------
> Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com
> Free/Busy: http://tinyurl.com/eric-cal
Re: Best way to unit test solr integration
Posted by Eric Pugh <ep...@opensourceconnections.com>.
So in the "building block" story you talked about, that sounds like an
integration (functional? user acceptance?) test.. And I would treat
Solr the same way you treat your database that you are storing model
objects in.
If in your tests you bring up a fresh version of the db, populate it
with tables etc, put in sample data, then you should do the same with
Solr. My guess is that you have a "test" database running, and
therefore you need a live supported "test" Solr. And the same
processes you use so that two functional tests don't step on each
others data in the database can be applied to Solr!
You can think of tweaking solr config changes as similar to tweaking
indexes in your db.. Both require Configuration Management to track
those changes, ensure they are deployed, and don't regress anything.
Let us know how you get on!
Eric
On Mar 27, 2009, at 12:50 PM, Joe Pollard wrote:
> Thanks for the tips, I like the suggestion of testing the document
> and query generation without having solr involved. That seems like
> a more bite-sized unit; I think I'll do that.
>
> However, here's the test case that I'm considering where I'd like to
> have a live solr instance:
>
> During an exercise of optimizing our schema, I'm going to be making
> wholesale changes that I'd like to ensure don't break some portion
> of our app. It seems like a good method for this would be to write
> a test with the following steps: (arguably not a unit test, but a
> very valuable test indeed in our application)
> * take some defined model object generated at test time, store it in
> db
> * run it through our document creation code
> * submit it into solr
> * generate a query using our custom criteria-based generation code
> * ensure that the query returns the results as expected
> * flesh out the new model objects from the db using only the id
> fields returned from Solr
> * In the end, it would be expected to have model objects retrieved
> from the db that match model objects at the beginning of the test.
>
> These building blocks could be stacked in numerous ways to test
> almost all the different scenarios in which we use Solr.
>
> Also, when/if we start making solr config changes, I can ensure that
> they change nothing from my app's functional point of view (with the
> exception of ridding us of dreaded OOMs).
>
> Thanks,
> -Joe
>
> -----Original Message-----
> From: Eric Pugh [mailto:epugh@opensourceconnections.com]
> Sent: Friday, March 27, 2009 11:27 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Best way to unit test solr integration
>
> So my first thought is that "unit test + solr integration" is an
> oxymoron. In the sense that unit test implies the smallest functional
> unit, and solr integration implies multiple units working together.
>
> It sounds like you have two different tasks. the code that generate
> queies, you can test that without Solr. If you need to parse some
> sort of solr document to generate a query based on it, then mock up
> the query. A lot of folks will just use Solr to build a result set,
> and then save that on the filesystem. "my_big_result1.xml" and read
> it in and feed it to your code.
>
> On the other hand, for you code testing indexing and retrieval, again,
> if you can use the same approach to decouple what solr does from your
> code. Unless you've patched Solr, you shouldn't need to unit test
> Solr, Solr has very nice unit testing built in.
>
> On the other hand, if you are doing integration testing, where you
> want a more end to end view of your application, then you probably
> already have a "test" solr setup in your environment somewhere that
> you can rely on to use.
>
> Spinning up and shutting down Solr for tests can be done, and I can
> think of use cases for why you might want to do it, but it does incur
> a penalty of being more work. And you still need to validate that
> your "embedded/unit test" solr works the same as your integration/test
> environment Solr.
>
> Eric
>
>
>
> On Mar 27, 2009, at 11:59 AM, Joe Pollard wrote:
>
>> Hello,
>>
>> On our project, we have quite a bit of code used to generate Solr
>> queries, and I need to create some unit tests to ensure that these
>> continue to work. In addition, I need to generate some unit tests
>> that will test indexing and retrieval of certain documents, based on
>> our current schema and the application logic that generates the
>> indexable documents as well as generates the Solr queries.
>>
>> My question is - what's the best way for me to unit test our Solr
>> integration?
>>
>> I'd like to be able to spin up an embedded/in-memory solr, or that
>> failing just start one up as part of my test case setup, fill it
>> with interesting documents, and do some queries, comparing the
>> results to expected results.
>>
>> Are there wiki pages or other documented examples of doing this? It
>> seems rather straight-forward, but who knows, it may be dead simple
>> with some unknown feature.
>>
>> Thanks!
>> -Joe
>
> -----------------------------------------------------
> Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
> Free/Busy: http://tinyurl.com/eric-cal
>
>
>
>
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Free/Busy: http://tinyurl.com/eric-cal
RE: Best way to unit test solr integration
Posted by Joe Pollard <Jo...@bazaarvoice.com>.
Thanks for the tips, I like the suggestion of testing the document and query generation without having solr involved. That seems like a more bite-sized unit; I think I'll do that.
However, here's the test case that I'm considering where I'd like to have a live solr instance:
During an exercise of optimizing our schema, I'm going to be making wholesale changes that I'd like to ensure don't break some portion of our app. It seems like a good method for this would be to write a test with the following steps: (arguably not a unit test, but a very valuable test indeed in our application)
* take some defined model object generated at test time, store it in db
* run it through our document creation code
* submit it into solr
* generate a query using our custom criteria-based generation code
* ensure that the query returns the results as expected
* flesh out the new model objects from the db using only the id fields returned from Solr
* In the end, it would be expected to have model objects retrieved from the db that match model objects at the beginning of the test.
These building blocks could be stacked in numerous ways to test almost all the different scenarios in which we use Solr.
Also, when/if we start making solr config changes, I can ensure that they change nothing from my app's functional point of view (with the exception of ridding us of dreaded OOMs).
Thanks,
-Joe
-----Original Message-----
From: Eric Pugh [mailto:epugh@opensourceconnections.com]
Sent: Friday, March 27, 2009 11:27 AM
To: solr-user@lucene.apache.org
Subject: Re: Best way to unit test solr integration
So my first thought is that "unit test + solr integration" is an
oxymoron. In the sense that unit test implies the smallest functional
unit, and solr integration implies multiple units working together.
It sounds like you have two different tasks. the code that generate
queies, you can test that without Solr. If you need to parse some
sort of solr document to generate a query based on it, then mock up
the query. A lot of folks will just use Solr to build a result set,
and then save that on the filesystem. "my_big_result1.xml" and read
it in and feed it to your code.
On the other hand, for you code testing indexing and retrieval, again,
if you can use the same approach to decouple what solr does from your
code. Unless you've patched Solr, you shouldn't need to unit test
Solr, Solr has very nice unit testing built in.
On the other hand, if you are doing integration testing, where you
want a more end to end view of your application, then you probably
already have a "test" solr setup in your environment somewhere that
you can rely on to use.
Spinning up and shutting down Solr for tests can be done, and I can
think of use cases for why you might want to do it, but it does incur
a penalty of being more work. And you still need to validate that
your "embedded/unit test" solr works the same as your integration/test
environment Solr.
Eric
On Mar 27, 2009, at 11:59 AM, Joe Pollard wrote:
> Hello,
>
> On our project, we have quite a bit of code used to generate Solr
> queries, and I need to create some unit tests to ensure that these
> continue to work. In addition, I need to generate some unit tests
> that will test indexing and retrieval of certain documents, based on
> our current schema and the application logic that generates the
> indexable documents as well as generates the Solr queries.
>
> My question is - what's the best way for me to unit test our Solr
> integration?
>
> I'd like to be able to spin up an embedded/in-memory solr, or that
> failing just start one up as part of my test case setup, fill it
> with interesting documents, and do some queries, comparing the
> results to expected results.
>
> Are there wiki pages or other documented examples of doing this? It
> seems rather straight-forward, but who knows, it may be dead simple
> with some unknown feature.
>
> Thanks!
> -Joe
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Free/Busy: http://tinyurl.com/eric-cal
Re: Best way to unit test solr integration
Posted by Eric Pugh <ep...@opensourceconnections.com>.
So my first thought is that "unit test + solr integration" is an
oxymoron. In the sense that unit test implies the smallest functional
unit, and solr integration implies multiple units working together.
It sounds like you have two different tasks. the code that generate
queies, you can test that without Solr. If you need to parse some
sort of solr document to generate a query based on it, then mock up
the query. A lot of folks will just use Solr to build a result set,
and then save that on the filesystem. "my_big_result1.xml" and read
it in and feed it to your code.
On the other hand, for you code testing indexing and retrieval, again,
if you can use the same approach to decouple what solr does from your
code. Unless you've patched Solr, you shouldn't need to unit test
Solr, Solr has very nice unit testing built in.
On the other hand, if you are doing integration testing, where you
want a more end to end view of your application, then you probably
already have a "test" solr setup in your environment somewhere that
you can rely on to use.
Spinning up and shutting down Solr for tests can be done, and I can
think of use cases for why you might want to do it, but it does incur
a penalty of being more work. And you still need to validate that
your "embedded/unit test" solr works the same as your integration/test
environment Solr.
Eric
On Mar 27, 2009, at 11:59 AM, Joe Pollard wrote:
> Hello,
>
> On our project, we have quite a bit of code used to generate Solr
> queries, and I need to create some unit tests to ensure that these
> continue to work. In addition, I need to generate some unit tests
> that will test indexing and retrieval of certain documents, based on
> our current schema and the application logic that generates the
> indexable documents as well as generates the Solr queries.
>
> My question is - what's the best way for me to unit test our Solr
> integration?
>
> I'd like to be able to spin up an embedded/in-memory solr, or that
> failing just start one up as part of my test case setup, fill it
> with interesting documents, and do some queries, comparing the
> results to expected results.
>
> Are there wiki pages or other documented examples of doing this? It
> seems rather straight-forward, but who knows, it may be dead simple
> with some unknown feature.
>
> Thanks!
> -Joe
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Free/Busy: http://tinyurl.com/eric-cal