Question about custom Lucene filters and Solr

[Jon Bodner <jo...@it.com>]

Hello,

I'm interested in using Solr with a custom Lucene Filter (like the one described in section 6.4.1 of the Lucene In Action, Second Edition book).  I'd like to filter search results from a Lucene index against information stored in a relational database.  I don't want to move the relational database information into the search index, because it could change frequently.

I looked at writing my own custom Solr SearchComponent, but the documentation for those seems slim.  Is this the correct approach?  Is there another way?

Thanks, 

Jon

Re: Question about custom Lucene filters and Solr

[Jon Bodner <jo...@it.com>]

Hi Israel (et al),

I don't think that I need an Update Handler; I don't intend to change the values in the search index (in fact, the goal is to build a Lucene index with Hadoop and then point a Solr instance at it).

What I'm trying to do is split the document into two locations: one is the Lucene index, the other is a database table.  The searchable fields that don't change are in Lucene.  The searchable fields that do change are in the database.  What I would like to do is add something similar to the existing the queryFilter functionality.  Rather than have filter that are built from a Lucene query, I want to have filters that are built from some additional parameters that are passed in to Solr.

I'm looking at doing something in the process() method of a SearchComponent, but that seems to be too late, as the query would already exist and I'd have to modify the existing docList and docSet.  It'd be nice to do something that runs inside of the QueryComponent.

Thanks,

Jon

On Feb 16, 2010, at 4:51 PM, Israel Ekpo wrote:

> Hi Jon,
> 
> You will need to write a plugin
> 
> You will need custom Query parser and an Update Handler depending on what
> you are doing.
> 
> The implementation of an Update Handler or Update Request Processor is not
> recommended because it is considered to be advanced.
> 
> Take a look at the following links for more information
> 
> http://wiki.apache.org/solr/SolrPlugins
> 
> http://wiki.apache.org/solr/UpdateRequestProcessor
> 
> http://lucene.apache.org/solr/api/org/apache/solr/update/UpdateHandler.html
> 
> http://lucene.apache.org/solr/api/org/apache/solr/search/QParserPlugin.html
> 
> http://lucene.apache.org/solr/api/org/apache/solr/update/processor/UpdateRequestProcessor.html
> 
> On Tue, Feb 16, 2010 at 2:43 PM, Jon Bodner <jo...@it.com> wrote:
> 
>> Hello,
>> 
>> I'm interested in using Solr with a custom Lucene Filter (like the one
>> described in section 6.4.1 of the Lucene In Action, Second Edition book).
>> I'd like to filter search results from a Lucene index against information
>> stored in a relational database.  I don't want to move the relational
>> database information into the search index, because it could change
>> frequently.
>> 
>> I looked at writing my own custom Solr SearchComponent, but the
>> documentation for those seems slim.  Is this the correct approach?  Is there
>> another way?
>> 
>> Thanks,
>> 
>> Jon
>> 
> 
> 
> 
> -- 
> "Good Enough" is not good enough.
> To give anything less than your best is to sacrifice the gift.
> Quality First. Measure Twice. Cut Once.
> http://www.israelekpo.com/

Re: Question about custom Lucene filters and Solr

[Israel Ekpo <is...@gmail.com>]

Hi Jon,

You will need to write a plugin

You will need custom Query parser and an Update Handler depending on what
you are doing.

The implementation of an Update Handler or Update Request Processor is not
recommended because it is considered to be advanced.

Take a look at the following links for more information

http://wiki.apache.org/solr/SolrPlugins

http://wiki.apache.org/solr/UpdateRequestProcessor

http://lucene.apache.org/solr/api/org/apache/solr/update/UpdateHandler.html

http://lucene.apache.org/solr/api/org/apache/solr/search/QParserPlugin.html

http://lucene.apache.org/solr/api/org/apache/solr/update/processor/UpdateRequestProcessor.html

On Tue, Feb 16, 2010 at 2:43 PM, Jon Bodner <jo...@it.com> wrote:

> Hello,
>
> I'm interested in using Solr with a custom Lucene Filter (like the one
> described in section 6.4.1 of the Lucene In Action, Second Edition book).
>  I'd like to filter search results from a Lucene index against information
> stored in a relational database.  I don't want to move the relational
> database information into the search index, because it could change
> frequently.
>
> I looked at writing my own custom Solr SearchComponent, but the
> documentation for those seems slim.  Is this the correct approach?  Is there
> another way?
>
> Thanks,
>
> Jon
>



-- 
"Good Enough" is not good enough.
To give anything less than your best is to sacrifice the gift.
Quality First. Measure Twice. Cut Once.
http://www.israelekpo.com/

Re: Question about custom Lucene filters and Solr

[Chris Hostetter <ho...@fucit.org>]

: I'm interested in using Solr with a custom Lucene Filter (like the one 
: described in section 6.4.1 of the Lucene In Action, Second Edition 
: book).  I'd like to filter search results from a Lucene index against 
: information stored in a relational database.  I don't want to move the 
: relational database information into the search index, because it could 
: change frequently.

One approach would be to subclass the either the QueryComponent or the 
SearchHandler and generate a DocSet from your external data source that 
you specify as a filter before QueryComponent executes the search.

The other possibility would be to write a QParserPlugin that generated a 
Query which when executed uses the index only to get the 
fieldCache for docId->uniqueKey mappings, and decides what documents to 
match based entirely on your external datasource.  then you can use the 
"fq" param to trigger your custom parser and Solr will use the DocSet 
produced as a filter.


-Hoss