You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Geoffrey Young (JIRA)" <ji...@apache.org> on 2008/06/17 19:51:45 UTC
[jira] Updated: (SOLR-14) Add the ability to preserve the original
term when using WordDelimiterFilter
[ https://issues.apache.org/jira/browse/SOLR-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Geoffrey Young updated SOLR-14:
-------------------------------
Attachment: SOLR-14.patch
ok, I've given this a shot. I'm an an open-source guy, even an ASF guy, but not a java guy, so forgive my code ;)
the patch should apply cleanly to current trunk. the last patch before mine still had some issues that needed to be worked through wrt term duplication. this patch should work a bit better.
all current tests pass when adjusted to 'preserveOriginal=0' (default behavior, same as 1.2). I looked at augmenting the current tests for WDF and 'preserveOriginal=1' but it's beyond my current java abilities.
installing the patch and running the analyzer yields stuff like this:
foo => foo
foo-bar => foo-bar foo bar foobar
foo-bar baz => foo-bar foo bar foobar baz
foo! => foo foo!
which seems reasonable to me.
a little shepherding would be awesome.
thanks
--Geoff
> Add the ability to preserve the original term when using WordDelimiterFilter
> ----------------------------------------------------------------------------
>
> Key: SOLR-14
> URL: https://issues.apache.org/jira/browse/SOLR-14
> Project: Solr
> Issue Type: Improvement
> Components: search
> Reporter: Richard "Trey" Hyde
> Attachments: SOLR-14.patch, TokenizerFactory.java, WordDelimiterFilter.patch, WordDelimiterFilter.patch
>
>
> When doing prefix searching, you need to hang on to the original term othewise you'll miss many matches you should be making.
> Data: ABC-12345
> WordDelimiterFitler may change this into
> ABC 12345 ABC12345
> A user may enter a search such as
> ABC\-123*
> Which will fail to find a match given the above scenario.
> The attached patch will allow the use of the "preserveOriginal" option to WordDelimiterFilter and will analyse as
> ABC 12345 ABC12345 ABC-12345
> in which case we will get a postive match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.