You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Avlesh Singh <av...@gmail.com> on 2010/01/04 05:54:11 UTC
Rules engine and Solr
I have a Solr (version 1.3) powered search server running in production.
Search is keyword driven is supported using custom fields and tokenizers.
I am planning to build a rules engine on top search. The rules are database
driven and can't be stored inside solr indexes. These rules would ultimately
two do things -
1. Change the order of Lucene hits.
2. Add/remove some results to/from the Lucene hits.
What should be my starting point? Custom search handler?
Cheers
Avlesh
Re: Rules engine and Solr
Posted by Avlesh Singh <av...@gmail.com>.
Thanks for the response, Shalin. I am still in two minds over doing it
"inside" Solr versus "outside".
I'll get back with more questions, if any.
Cheers
Avlesh
On Mon, Jan 4, 2010 at 5:11 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> On Mon, Jan 4, 2010 at 10:24 AM, Avlesh Singh <av...@gmail.com> wrote:
>
> > I have a Solr (version 1.3) powered search server running in production.
> > Search is keyword driven is supported using custom fields and tokenizers.
> >
> > I am planning to build a rules engine on top search. The rules are
> database
> > driven and can't be stored inside solr indexes. These rules would
> > ultimately
> > two do things -
> >
> > 1. Change the order of Lucene hits.
> >
>
> A Lucene FieldComparator is what you'd need. The QueryElevationComponent
> uses this technique.
>
>
> > 2. Add/remove some results to/from the Lucene hits.
> >
> >
> This is a bit more tricky. If you will always have a very limited number of
> docs to add or remove, it may be best to change the query itself to include
> or exclude them (i.e. add fq). Otherwise you'd need to write a custom
> Collector (see DocSetCollector) and change SolrIndexSearcher to use it. We
> are planning to modify SolrIndexSearcher to allow custom collectors soon
> for
> field collapsing but for now you will have to modify it.
>
>
> > What should be my starting point? Custom search handler?
> >
> >
> A custom SearchComponent which extends/overrides QueryComponent will do the
> job.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: Rules engine and Solr
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Jan 4, 2010 at 10:24 AM, Avlesh Singh <av...@gmail.com> wrote:
> I have a Solr (version 1.3) powered search server running in production.
> Search is keyword driven is supported using custom fields and tokenizers.
>
> I am planning to build a rules engine on top search. The rules are database
> driven and can't be stored inside solr indexes. These rules would
> ultimately
> two do things -
>
> 1. Change the order of Lucene hits.
>
A Lucene FieldComparator is what you'd need. The QueryElevationComponent
uses this technique.
> 2. Add/remove some results to/from the Lucene hits.
>
>
This is a bit more tricky. If you will always have a very limited number of
docs to add or remove, it may be best to change the query itself to include
or exclude them (i.e. add fq). Otherwise you'd need to write a custom
Collector (see DocSetCollector) and change SolrIndexSearcher to use it. We
are planning to modify SolrIndexSearcher to allow custom collectors soon for
field collapsing but for now you will have to modify it.
> What should be my starting point? Custom search handler?
>
>
A custom SearchComponent which extends/overrides QueryComponent will do the
job.
--
Regards,
Shalin Shekhar Mangar.
Re: Rules engine and Solr
Posted by Avlesh Singh <av...@gmail.com>.
Thanks for the revert, Ravi.
I am currently working on some of kind rules in front
> (application side) of our solr instance. These rules are more application
> specific and are not general. Like deciding which fields to facet, which
> fields to return in response, which fields to highlight, boost value for
> each field (both at query time and at index time).
> The approach I have taken is to define a database table which
> holds these fields parameters. Which are then interpreted by my application
> to decide the query to be sent to Solr. This allow tweaking the Solr fields
> on the fly and hence influence the search results.
>
I guess, this is the usual usage of solr server. In my case this is no
different. Search queries have a personalized experience, which means
behaviors for facets, highlighting etc .. are customizable. We pull it off
using databases and java data structures.
I will be interested to hear from you about the "Kind" of rules you talk
> about and your approach towards it. Are these "Rules" like a regular
> expression that when matched with the "user query", execute a specific
> "solr
> query" ?
>
http://en.wikipedia.org/wiki/Business_rules_engine
Cheers
Avlesh
On Wed, Jan 6, 2010 at 12:12 PM, Ravi Gidwani <ra...@gmail.com>wrote:
> Avlesh:
> I am currently working on some of kind rules in front
> (application side) of our solr instance. These rules are more application
> specific and are not general. Like deciding which fields to facet, which
> fields to return in response, which fields to highlight, boost value for
> each field (both at query time and at index time).
> The approach I have taken is to define a database table which
> holds these fields parameters. Which are then interpreted by my application
> to decide the query to be sent to Solr. This allow tweaking the Solr fields
> on the fly and hence influence the search results.
>
> I will be interested to hear from you about the "Kind" of rules you talk
> about and your approach towards it. Are these "Rules" like a regular
> expression that when matched with the "user query", execute a specific
> "solr
> query" ?
>
> ~Ravi
>
> On Tue, Jan 5, 2010 at 8:25 PM, Avlesh Singh <av...@gmail.com> wrote:
>
> > >
> > > Your question appears to be an "XY Problem" ... that is: you are
> dealing
> > > with "X", you are assuming "Y" will help you, and you are asking about
> > "Y"
> > > without giving more details about the "X" so that we can understand the
> > full
> > > issue. Perhaps the best solution doesn't involve "Y" at all? See Also:
> > > http://www.perlmonks.org/index.pl?node_id=542341
> > >
> > Hahaha, thats classic Hoss!
> > Thanks for introducing me to the XY problem. Had I known the two
> > completely,
> > I wouldn't have posted it on the mailing list. And I wasn't looking for a
> > "solution" either. Anyways, as I replied back earlier, I'll get back with
> > questions once I get more clarity.
> >
> > Cheers
> > Avlesh
> >
> > On Wed, Jan 6, 2010 at 2:02 AM, Chris Hostetter <
> hossman_lucene@fucit.org
> > >wrote:
> >
> > >
> > > : I am planning to build a rules engine on top search. The rules are
> > > database
> > > : driven and can't be stored inside solr indexes. These rules would
> > > ultimately
> > > : two do things -
> > > :
> > > : 1. Change the order of Lucene hits.
> > > : 2. Add/remove some results to/from the Lucene hits.
> > > :
> > > : What should be my starting point? Custom search handler?
> > >
> > > This smells like an XY problem ... can you elaborate on the types of
> > > rules/conditions/situations when you want #1 and #2 listed above to
> > > happen?
> > >
> > > http://people.apache.org/~hossman/#xyproblem<http://people.apache.org/%7Ehossman/#xyproblem>
> <http://people.apache.org/%7Ehossman/#xyproblem>
> > <http://people.apache.org/%7Ehossman/#xyproblem>
> > > XY Problem
> > >
> > > Your question appears to be an "XY Problem" ... that is: you are
> dealing
> > > with "X", you are assuming "Y" will help you, and you are asking about
> > "Y"
> > > without giving more details about the "X" so that we can understand the
> > > full issue. Perhaps the best solution doesn't involve "Y" at all?
> > > See Also: http://www.perlmonks.org/index.pl?node_id=542341
> > >
> > >
> > >
> > >
> > >
> > > -Hoss
> > >
> > >
> >
>
Re: Rules engine and Solr
Posted by Ravi Gidwani <ra...@gmail.com>.
Avlesh:
I am currently working on some of kind rules in front
(application side) of our solr instance. These rules are more application
specific and are not general. Like deciding which fields to facet, which
fields to return in response, which fields to highlight, boost value for
each field (both at query time and at index time).
The approach I have taken is to define a database table which
holds these fields parameters. Which are then interpreted by my application
to decide the query to be sent to Solr. This allow tweaking the Solr fields
on the fly and hence influence the search results.
I will be interested to hear from you about the "Kind" of rules you talk
about and your approach towards it. Are these "Rules" like a regular
expression that when matched with the "user query", execute a specific "solr
query" ?
~Ravi
On Tue, Jan 5, 2010 at 8:25 PM, Avlesh Singh <av...@gmail.com> wrote:
> >
> > Your question appears to be an "XY Problem" ... that is: you are dealing
> > with "X", you are assuming "Y" will help you, and you are asking about
> "Y"
> > without giving more details about the "X" so that we can understand the
> full
> > issue. Perhaps the best solution doesn't involve "Y" at all? See Also:
> > http://www.perlmonks.org/index.pl?node_id=542341
> >
> Hahaha, thats classic Hoss!
> Thanks for introducing me to the XY problem. Had I known the two
> completely,
> I wouldn't have posted it on the mailing list. And I wasn't looking for a
> "solution" either. Anyways, as I replied back earlier, I'll get back with
> questions once I get more clarity.
>
> Cheers
> Avlesh
>
> On Wed, Jan 6, 2010 at 2:02 AM, Chris Hostetter <hossman_lucene@fucit.org
> >wrote:
>
> >
> > : I am planning to build a rules engine on top search. The rules are
> > database
> > : driven and can't be stored inside solr indexes. These rules would
> > ultimately
> > : two do things -
> > :
> > : 1. Change the order of Lucene hits.
> > : 2. Add/remove some results to/from the Lucene hits.
> > :
> > : What should be my starting point? Custom search handler?
> >
> > This smells like an XY problem ... can you elaborate on the types of
> > rules/conditions/situations when you want #1 and #2 listed above to
> > happen?
> >
> > http://people.apache.org/~hossman/#xyproblem<http://people.apache.org/%7Ehossman/#xyproblem>
> <http://people.apache.org/%7Ehossman/#xyproblem>
> > XY Problem
> >
> > Your question appears to be an "XY Problem" ... that is: you are dealing
> > with "X", you are assuming "Y" will help you, and you are asking about
> "Y"
> > without giving more details about the "X" so that we can understand the
> > full issue. Perhaps the best solution doesn't involve "Y" at all?
> > See Also: http://www.perlmonks.org/index.pl?node_id=542341
> >
> >
> >
> >
> >
> > -Hoss
> >
> >
>
Re: Rules engine and Solr
Posted by Avlesh Singh <av...@gmail.com>.
>
> Your question appears to be an "XY Problem" ... that is: you are dealing
> with "X", you are assuming "Y" will help you, and you are asking about "Y"
> without giving more details about the "X" so that we can understand the full
> issue. Perhaps the best solution doesn't involve "Y" at all? See Also:
> http://www.perlmonks.org/index.pl?node_id=542341
>
Hahaha, thats classic Hoss!
Thanks for introducing me to the XY problem. Had I known the two completely,
I wouldn't have posted it on the mailing list. And I wasn't looking for a
"solution" either. Anyways, as I replied back earlier, I'll get back with
questions once I get more clarity.
Cheers
Avlesh
On Wed, Jan 6, 2010 at 2:02 AM, Chris Hostetter <ho...@fucit.org>wrote:
>
> : I am planning to build a rules engine on top search. The rules are
> database
> : driven and can't be stored inside solr indexes. These rules would
> ultimately
> : two do things -
> :
> : 1. Change the order of Lucene hits.
> : 2. Add/remove some results to/from the Lucene hits.
> :
> : What should be my starting point? Custom search handler?
>
> This smells like an XY problem ... can you elaborate on the types of
> rules/conditions/situations when you want #1 and #2 listed above to
> happen?
>
> http://people.apache.org/~hossman/#xyproblem<http://people.apache.org/%7Ehossman/#xyproblem>
> XY Problem
>
> Your question appears to be an "XY Problem" ... that is: you are dealing
> with "X", you are assuming "Y" will help you, and you are asking about "Y"
> without giving more details about the "X" so that we can understand the
> full issue. Perhaps the best solution doesn't involve "Y" at all?
> See Also: http://www.perlmonks.org/index.pl?node_id=542341
>
>
>
>
>
> -Hoss
>
>
Re: Rules engine and Solr
Posted by Chris Hostetter <ho...@fucit.org>.
: I am planning to build a rules engine on top search. The rules are database
: driven and can't be stored inside solr indexes. These rules would ultimately
: two do things -
:
: 1. Change the order of Lucene hits.
: 2. Add/remove some results to/from the Lucene hits.
:
: What should be my starting point? Custom search handler?
This smells like an XY problem ... can you elaborate on the types of
rules/conditions/situations when you want #1 and #2 listed above to
happen?
http://people.apache.org/~hossman/#xyproblem
XY Problem
Your question appears to be an "XY Problem" ... that is: you are dealing
with "X", you are assuming "Y" will help you, and you are asking about "Y"
without giving more details about the "X" so that we can understand the
full issue. Perhaps the best solution doesn't involve "Y" at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341
-Hoss