You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-commits@lucene.apache.org by ot...@apache.org on 2007/03/16 20:28:48 UTC
svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt
example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml
lib/lucene-spellchecker-2.2-dev.jar
src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
Author: otis
Date: Fri Mar 16 12:28:47 2007
New Revision: 519107
URL: http://svn.apache.org/viewvc?view=rev&rev=519107
Log:
- SOLR-81: SpellCheckerRequentHandler
Added:
lucene/solr/trunk/example/exampledocs/spellchecher.xml
lucene/solr/trunk/lib/lucene-spellchecker-2.2-dev.jar (with props)
lucene/solr/trunk/src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
Modified:
lucene/solr/trunk/CHANGES.txt
lucene/solr/trunk/example/solr/conf/solrconfig.xml
Modified: lucene/solr/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?view=diff&rev=519107&r1=519106&r2=519107
==============================================================================
--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Fri Mar 16 12:28:47 2007
@@ -120,7 +120,11 @@
allows for specifying the amount of default slop to use when parsing
explicit phrase queries from the user.
(Adam Hiatt via hossman)
-
+
+16. SOLR-81: SpellCheckerRequestHandler that uses the SpellChecker from
+ the Lucene contrib.
+ (Otis Gospodnetic and Adam Hiatt)
+
Changes in runtime behavior
1. Highlighting using DisMax will only pick up terms from the main
user query, not boost or filter queries (klaas).
Added: lucene/solr/trunk/example/exampledocs/spellchecher.xml
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/example/exampledocs/spellchecher.xml?view=auto&rev=519107
==============================================================================
--- lucene/solr/trunk/example/exampledocs/spellchecher.xml (added)
+++ lucene/solr/trunk/example/exampledocs/spellchecher.xml Fri Mar 16 12:28:47 2007
@@ -0,0 +1,42 @@
+<add>
+ <doc>
+ <field name="id">1</field>
+ <field name="word">ipod</field>
+ </doc>
+ <doc>
+ <field name="id">2</field>
+ <field name="word">ipod nano</field>
+ </doc>
+ <doc>
+ <field name="id">3</field>
+ <field name="word">ipod video</field>
+ </doc>
+ <doc>
+ <field name="id">4</field>
+ <field name="word">ipod shuffle</field>
+ </doc>
+ <doc>
+ <field name="id">5</field>
+ <field name="word">wii</field>
+ </doc>
+ <doc>
+ <field name="id">6</field>
+ <field name="word">blackberry</field>
+ </doc>
+ <doc>
+ <field name="id">7</field>
+ <field name="word">blackjack</field>
+ </doc>
+ <doc>
+ <field name="id">8</field>
+ <field name="word">creative</field>
+ </doc>
+ <doc>
+ <field name="id">9</field>
+ <field name="word">creative labs</field>
+ </doc>
+ <doc>
+ <field name="id">10</field>
+ <field name="word">creative zen</field>
+ </doc>
+</add>
Modified: lucene/solr/trunk/example/solr/conf/solrconfig.xml
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/example/solr/conf/solrconfig.xml?view=diff&rev=519107&r1=519106&r2=519107
==============================================================================
--- lucene/solr/trunk/example/solr/conf/solrconfig.xml (original)
+++ lucene/solr/trunk/example/solr/conf/solrconfig.xml Fri Mar 16 12:28:47 2007
@@ -341,7 +341,24 @@
2<-1 5<-2 6<90%
</str>
</requestHandler>
-
+
+
+ <!-- SpellCheckerRequestHandler takes in a word (or several words) as the
+ value of the "q" parameter and returns a list of alternative spelling
+ suggestions. If invoked with a ...&cmd=rebuild, it will rebuild the
+ spellchecker index.
+ -->
+ <requestHandler name="spellchecker" class="solr.SpellCheckerRequestHandler">
+ <!-- default values for query parameters -->
+ <lst name="defaults">
+ <int name="suggestionCount">1</int>
+ <str name="version">1.0</str>
+ </lst>
+ <!-- main init params for handler -->
+ <str name="spellcheckerIndexDir">/home/otis/dev/repos/lucene/solr/trunk/example/solr/data/index</str>
+ <str name="termSourceField">word</str>
+ </requestHandler>
+
<!-- Standard update plugin. If we put this on /update, it will get all the new goodness -->
<requestHandler name="/update/xml" class="solr.XmlUpdateRequestHandler" >
Added: lucene/solr/trunk/lib/lucene-spellchecker-2.2-dev.jar
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/lib/lucene-spellchecker-2.2-dev.jar?view=auto&rev=519107
==============================================================================
Binary file - no diff available.
Propchange: lucene/solr/trunk/lib/lucene-spellchecker-2.2-dev.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: lucene/solr/trunk/src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/request/SpellCheckerRequestHandler.java?view=auto&rev=519107
==============================================================================
--- lucene/solr/trunk/src/java/org/apache/solr/request/SpellCheckerRequestHandler.java (added)
+++ lucene/solr/trunk/src/java/org/apache/solr/request/SpellCheckerRequestHandler.java Fri Mar 16 12:28:47 2007
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.request;
+
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.search.spell.SpellChecker;
+import org.apache.lucene.store.FSDirectory;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.RequestHandlerBase;
+import org.apache.solr.util.NamedList;
+
+import java.io.IOException;
+import java.net.URL;
+import java.util.Arrays;
+
+/**
+ * Takes a string (e.g. a query string) as the value of the "q" parameter
+ * and looks up alternative spelling suggestions in the spellchecker.
+ * The spellchecker used by this handler is the Lucene contrib SpellChecker.
+ * @see http://wiki.apache.org/jakarta-lucene/SpellChecker
+ *
+ * @author Otis Gospodnetic
+ */
+public class SpellCheckerRequestHandler extends RequestHandlerBase {
+
+ private static SpellChecker spellChecker;
+
+ /*
+ * From http://wiki.apache.org/jakarta-lucene/SpellChecker
+ * If reader and restrictToField are both not null:
+ * 1. The returned words are restricted only to the words presents in the field
+ * "restrictToField "of the Lucene Index "reader".
+ *
+ * 2. The list is also sorted with a second criterium: the popularity (the
+ * frequence) of the word in the user field.
+ *
+ * 3. If "onlyMorePopular" is true and the mispelled word exist in the user field,
+ * return only the words more frequent than this.
+ *
+ */
+ private static IndexReader reader = null;
+ private String restrictToField = null;
+ private boolean onlyMorePopular = false;
+
+ private String spellcheckerIndexDir;
+
+ public void init(NamedList args) {
+ super.init( args );
+ spellcheckerIndexDir = invariants.get("spellcheckerIndexDir");
+ try {
+ spellChecker = new SpellChecker(FSDirectory.getDirectory(spellcheckerIndexDir));
+ } catch (IOException e) {
+ throw new RuntimeException("Cannot open SpellChecker index", e);
+ }
+ }
+
+ public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp)
+ throws Exception {
+ SolrParams p = req.getParams();
+ String words = p.get("q");
+
+ System.out.println(getDescription());
+ int numSug = 5;
+ String[] suggestions = spellChecker.suggestSimilar(words, numSug,
+ reader, restrictToField, onlyMorePopular);
+
+ rsp.add("suggestions", Arrays.asList(suggestions));
+ }
+
+ //////////////////////// SolrInfoMBeans methods //////////////////////
+
+ public String getVersion() {
+ return SolrCore.version;
+ }
+
+ public String getDescription() {
+ return "The SpellChecker Solr request handler for SpellChecker index: " + spellcheckerIndexDir;
+ }
+
+ public String getSourceId() {
+ return "$Id$";
+ }
+
+ public String getSource() {
+ return "$URL$";
+ }
+
+ public URL[] getDocs() {
+ return null;
+ }
+}
Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
Posted by Ryan McKinley <ry...@gmail.com>.
A few trivial comments on SpellCheckerRequestHandler:
1. Should it go in "org.apache.solr.handler" rather then
"org.apache.solr.request"? As the number of special handlers
increases, it seems good to have them semi-isolated.
2. replace tabs with two spaces
3. The file needs the svn props set
4. getVersion() should return "$Revision: 501512 $"; rather then
SolrCore.version
nit pick nit pick nit pick. otherwise looks good!
Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Mar 16, 2007, at 3:28 PM, otis@apache.org wrote:
> Author: otis
> lucene/solr/trunk/example/exampledocs/spellchecher.xml
that should be spellchecker, not spellchecher.
Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/SpellCheckerRequestHandler.java
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Mar 16, 2007, at 3:28 PM, otis@apache.org wrote:
> + <str name="spellcheckerIndexDir">/home/otis/dev/repos/lucene/
> solr/trunk/example/solr/data/index</str>
And this needs to be changed to something globally relevant. What
triggers the creation of this index? With this part of the example,
we ought to flesh it out so that it works for anyone trying out Solr,
right?
> + public void init(NamedList args) {
> + super.init( args );
> + spellcheckerIndexDir = invariants.get
> ("spellcheckerIndexDir");
> + try {
> + spellChecker = new SpellChecker
> (FSDirectory.getDirectory(spellcheckerIndexDir));
> + } catch (IOException e) {
> + throw new RuntimeException("Cannot open SpellChecker
> index", e);
> + }
> + }
If this index were updated, what would trigger it to get refreshed by
this request handler?