You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Tim Allison (JIRA)" <ji...@apache.org> on 2013/06/06 13:26:20 UTC
[jira] [Comment Edited] (LUCENE-5033) SlowFuzzyQuery appears to
fail with edit distance >=3 in some cases
[ https://issues.apache.org/jira/browse/LUCENE-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13676919#comment-13676919 ]
Tim Allison edited comment on LUCENE-5033 at 6/6/13 11:25 AM:
--------------------------------------------------------------
Robert,
I agree that it appears to, but if you want a distance > 2, the current levenshtein automaton doesn't allow that (http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html?showComment=1303598602291#c3051732466052117784). The classic QueryParser silently converts distances > 2 to 2.
If I understand SlowFuzzyQuery correctly, it uses the levenshtein automaton for distances <= 2, but it runs brute force if the distance is > 2.
My personal preference would be to undeprecate SlowFuzzyQuery (certainly leave it in the sandbox) because it offers a capability that the current levenshtein automaton doesn't. In cases where the indices are very large, it wouldn't make sense to expose distance > 2 capability; but on small to medium indices, there are use cases that require it.
was (Author: tallison@mitre.org):
I agree that it appears to, but if you want a distance > 2, the current levenshtein automaton doesn't allow that (http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html?showComment=1303598602291#c3051732466052117784). The classic QueryParser silently converts distances > 2 to 2.
If I understand SlowFuzzyQuery correctly, it uses the levenshtein automaton for distances <= 2, but it runs brute force if the distance is > 2.
My personal preference would be to undeprecate SlowFuzzyQuery (certainly leave it in the sandbox) because it offers a capability that the current levenshtein automaton doesn't. In cases where the indices are very large, it wouldn't make sense to expose distance > 2 capability; but on small to medium indices, there are use cases that require it.
> SlowFuzzyQuery appears to fail with edit distance >=3 in some cases
> -------------------------------------------------------------------
>
> Key: LUCENE-5033
> URL: https://issues.apache.org/jira/browse/LUCENE-5033
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/other
> Affects Versions: 4.3
> Reporter: Tim Allison
> Priority: Minor
> Attachments: LUCENE-5033.patch
>
>
> Levenshtein edit btwn "monday" and "montugu" should be 4. The following shows a query with "sim" set to 3, and there is a hit.
> public void testFuzzinessLong2() throws Exception {
> Directory directory = newDirectory();
> RandomIndexWriter writer = new RandomIndexWriter(random(), directory);
> addDoc("monday", writer);
>
> IndexReader reader = writer.getReader();
> IndexSearcher searcher = newSearcher(reader);
> writer.close();
> SlowFuzzyQuery query;
> query = new SlowFuzzyQuery(new Term("field", "montugu"), 3, 0);
> ScoreDoc[] hits = searcher.search(query, null, 1000).scoreDocs;
> assertEquals(0, hits.length);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org