Seeking Anecdotes: Solr Plugins

[Chris Hostetter <ho...@fucit.org>]

Hey everybody, I'll be giving a talk called "Apache Solr: Beyond the Box" 
at ApacheCon this year, which will focus on the how/when/why of 
writing Solr Plugins...

 	http://us.apachecon.com/c/acus2008/sessions/10

I've got several use cases I can refer to for examples, both from my day 
job and from public projects (DIH, Local Solr, etc.) but I'm hoping to be 
able to provide even broader examples of the types of "niche" plugins 
people have written for specific purposes, and what their motivations 
where for doing so (as opposed to doing alternate logic in their client).

If people would like to reply to this thread to share a little bit of 
information about their experiences, that would be awesome -- in addition 
to helping me write my slides, it would help other Solr users be aware of 
what's possible (and help Solr developers be more aware of how our users 
are taking advantage of the APIs).

If there are reasons why you can't publicly disclose the details of 
how you are using Solr, but you wouldn't mind sharing some of the info 
with me directly on the grounds that I keep you (and your company) 
anonymous you can also feel free to email me directly.



-Hoss

Re: Seeking Anecdotes: Solr Plugins

[Mike Klaas <mi...@gmail.com>]

On 22-Jul-08, at 4:34 PM, Chris Hostetter wrote:

>
> Hey everybody, I'll be giving a talk called "Apache Solr: Beyond the  
> Box" at ApacheCon this year, which will focus on the how/when/why of  
> writing Solr Plugins...
>
> 	http://us.apachecon.com/c/acus2008/sessions/10
>
> I've got several use cases I can refer to for examples, both from my  
> day job and from public projects (DIH, Local Solr, etc.) but I'm  
> hoping to be able to provide even broader examples of the types of  
> "niche" plugins people have written for specific purposes, and what  
> their motivations where for doing so (as opposed to doing alternate  
> logic in their client).
>
> If people would like to reply to this thread to share a little bit  
> of information about their experiences, that would be awesome -- in  
> addition to helping me write my slides, it would help other Solr  
> users be aware of what's possible (and help Solr developers be more  
> aware of how our users are taking advantage of the APIs).


analysis:
  - a WDF that handles some special-case tokens (e.g., '.NET') and  
that doesn't split tokens that are a series of small components (e.g.,  
r2d2) for performance
  - analyzers to incorporate payloads
  - custom token splitters

highlighting:
  - custom scorer, fragmenter
  - request handler for highlighting specific docs without querying

custom queries/scoring:
  - a "proximity" query that can handle cases where only a part of a  
query is proximate
  - a query similar to CustomScoreQuery that allows the arbitrary  
combination of query scorers (e.g., product), but allows any Query  
subclass (not just a ValueSourceQuery).
  - a heavy modified version of dismax that incorporates the above  
query types along with payloads and other features like query-injected  
filter queries.  This type of extension is largely obsolete with  
QueryComponents

Let me know if you want more detail--most of this is relative to a  
somewhat older version of Solr, so it might not all apply.

cheers,
-Mike

Re: Seeking Anecdotes: Solr Plugins

[Grant Ingersoll <gs...@apache.org>]

I imagined the AnalysisRequestHandler might be useful for serving up  
vectors for things like machine learning experiments or just as a  
server that does analysis.  Also, someone on dev the other day talked  
about one that brought back the vectors w/ TF/IDF info as well.  I  
think Solr could make for quite a nice ML server.

Also, for Taming Text (again not there yet, but in the works and  
hopefully done in time for your talk :-)  http://manning.com/ 
ingersoll/), plan to have a Solr server that functions with OpenNLP (http://opennlp.sourceforge.net 
) and some other open source tools and can do things like named entity  
recognition, clustering and classification and make all of those  
things searchable.  Most (if not all) of these things will be done as  
some type of plugin (ReqHandler, SearchComponent, custom analyzers/ 
token filters, etc.).

Bigger question, though, is why are you preparing for ApacheCon so far  
in advance?  ;-)

-Grant

On Jul 22, 2008, at 7:34 PM, Chris Hostetter wrote:

>
> Hey everybody, I'll be giving a talk called "Apache Solr: Beyond the  
> Box" at ApacheCon this year, which will focus on the how/when/why of  
> writing Solr Plugins...
>
> 	http://us.apachecon.com/c/acus2008/sessions/10
>
> I've got several use cases I can refer to for examples, both from my  
> day job and from public projects (DIH, Local Solr, etc.) but I'm  
> hoping to be able to provide even broader examples of the types of  
> "niche" plugins people have written for specific purposes, and what  
> their motivations where for doing so (as opposed to doing alternate  
> logic in their client).
>
> If people would like to reply to this thread to share a little bit  
> of information about their experiences, that would be awesome -- in  
> addition to helping me write my slides, it would help other Solr  
> users be aware of what's possible (and help Solr developers be more  
> aware of how our users are taking advantage of the APIs).
>
> If there are reasons why you can't publicly disclose the details of  
> how you are using Solr, but you wouldn't mind sharing some of the  
> info with me directly on the grounds that I keep you (and your  
> company) anonymous you can also feel free to email me directly.
>
>
>
> -Hoss
>