You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Geoffrey Young (JIRA)" <ji...@apache.org> on 2008/06/17 19:51:45 UTC

[jira] Updated: (SOLR-14) Add the ability to preserve the original term when using WordDelimiterFilter

     [ https://issues.apache.org/jira/browse/SOLR-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Geoffrey Young updated SOLR-14:
-------------------------------

    Attachment: SOLR-14.patch

ok, I've given this a shot.  I'm an an open-source guy, even an ASF guy, but not a java guy, so forgive my code ;)

the patch should apply cleanly to current trunk.  the last patch before mine still had some issues that needed to be worked through wrt term duplication.  this patch should work a bit better.

all current tests pass when adjusted to 'preserveOriginal=0' (default behavior, same as 1.2).  I looked at augmenting the current tests for WDF and 'preserveOriginal=1' but it's beyond my current java abilities.

installing the patch and running the analyzer yields stuff like this:

  foo => foo
  foo-bar => foo-bar foo bar foobar
  foo-bar baz => foo-bar foo bar foobar baz
  foo! => foo foo!

which seems reasonable to me.

a little shepherding would be awesome.

thanks

--Geoff


> Add the ability to preserve the original term when using WordDelimiterFilter
> ----------------------------------------------------------------------------
>
>                 Key: SOLR-14
>                 URL: https://issues.apache.org/jira/browse/SOLR-14
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>            Reporter: Richard "Trey" Hyde
>         Attachments: SOLR-14.patch, TokenizerFactory.java, WordDelimiterFilter.patch, WordDelimiterFilter.patch
>
>
> When doing prefix searching, you need to hang on to the original term othewise you'll miss many matches you should be making.
> Data: ABC-12345
> WordDelimiterFitler may change this into
> ABC 12345 ABC12345
> A user may enter a search such as 
>  ABC\-123*
> Which will fail to find a match given the above scenario.
> The attached patch will allow the use of the "preserveOriginal" option to WordDelimiterFilter and will analyse as
> ABC 12345 ABC12345  ABC-12345 
> in which case we will get a postive match.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.