Re: dash-words

[Martin Braun <mb...@uni-hd.de>]

Hi Yonik,

>> So a Phrase search to "The xmen story" will fail. With a slop of 1 the
>> doc will be found.
>>
>> But when generating the query I won't know when to use a slop. So adding
>> slops isn't a nice solution.
> 
> If you can't tolerate slop, this is a problem.

I use the WordDelimiterFilter now without slop, because in other cases
it's an amelioration. But I (or better my app) stumbled now over a non
Phrase Query:

If I am searching for a title named (sorry for the german example).

"lage der arbeiterjugend in westberlin" (indexed with
WordDelimiterFilter + lowercase)

with a query like this +arbeiterjugend +west-berlin I get no results.

org.apache.lucene.queryParser.QueryParser.parse makes this query (with
WordDelimiterFilter) with Default QueryParser.AND_OPERATOR:

+titel:arbeiterjugend +titel:"west (berlin westberlin)"

with +arbeiterjugend +westberlin I get the result.

It seems that the synonyms don't work with the query. How do you solve
this in Solr? Do I have to build a TermQuery?

thanks in advance,

martin







---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: dash-words

[Chris Hostetter <ho...@fucit.org>]

: with a query like this +arbeiterjugend +west-berlin I get no results.
:
: org.apache.lucene.queryParser.QueryParser.parse makes this query (with
: WordDelimiterFilter) with Default QueryParser.AND_OPERATOR:
:
: +titel:arbeiterjugend +titel:"west (berlin westberlin)"
:
: with +arbeiterjugend +westberlin I get the result.
:
: It seems that the synonyms don't work with the query. How do you solve
: this in Solr? Do I have to build a TermQuery?

First off, when using WordDelimiterFilter it's generally a good idea to
use a slightly differnet configuration of the Filter in your indexng
analyzer then in your query analyzer -- this is discussed a bit inthe
wiki...

http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#head-1c9b83870ca7890cd73b193cefed83c283339089
http://wiki.apache.org/solr/SolrRelevancyCookbook#head-353fcfa33e5c4a0a5959aa3d8d33c5a3a61f2683

...this can help avoid situations like you describe.

but in general, what you are running into is a general constraint of the
way Analyzers can produce tokens with a "zero gap" indicating that they
occupy the same spot as the previous token, but there is no way for the
analyzer to indicate that a sequence of 1 or more tokens occupies the same
space as another sequence of 1 or more tokens.  so when QUeryParser asks
the analyzer to make a token stream out of "west-berlin" the analyzer has
no way to return a token stream that can easily be recognized as [ [[west]
[berlin]] or [westberlin] ].

this does in fact prove to be a large problem when dealing with "multi
word synonyms" (also discussed in the wiki mentioned above) but can
generally be dealt with in the WordDeliminterFilter.


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org