customer request handler doesn't envok the query tokenization chain

[Yu-Hui Jin <yu...@gmail.com>]

hi, there,

Here's a question regarding the relationship of a customer request handler
and a customer field type.

Let's say we defined a customer filed type that when querying and indexing,
the solr.LowerCaseFilterFactory is used as the last filter to low-case all
letters.  In the Analysis UI, we found tokenization is working correctly.

We also defined  a custom request handler which always creates a boolean
query that ANDs all tokens for fielded queries (we overrided the
"getFieldQuery" method only).  Now the problem occurs that when we use the
custom request handler to handle queries. we only get results for low-case
query string,  but not for the upper-case version.   (This is true when I
turned the "debugQuery" flag on and I saw the query string hasn't been
low-cased)  If we switch to use the Standard request handler, we don't have
the problem.   It seems that when using the customer request handler, the
tokenizer/filters chain is not executed (since the low-case filter is not
applied).   I'm not sure why.

Anyone knows what I'm missing here?


many thanks,

-Hui

Re: customer request handler doesn't envok the query tokenization chain

[Yonik Seeley <yo...@apache.org>]

On 11/5/07, Yu-Hui Jin <yu...@gmail.com> wrote:
> Just curious, does the default operator  ( "AND" or "OR") specify the
> relationship between a field/value component or between the tokens of the
> same field/value componenet?

between any clauses in a boolean query.

> e.g. for a query like this:
>
> field1:abc  field2:xyz
>
> does the operator   connect field1:abc and field2:xyz , or it connects  the
> tokens from "abc" and "xyz" for their respective field?

These are two different query clauses (the fieldnames don't matter).
If the default operator is "OR", then it will be interpreted as
field1:abc OR field2:xyz  (both optional)
if the default operator is set to "AND" then it will be
field1:abc AND field2:xyz  (both required)

-Yonik

Re: customer request handler doesn't envok the query tokenization chain

[Yu-Hui Jin <yu...@gmail.com>]

Thanks, Yonik.

Just curious, does the default operator  ( "AND" or "OR") specify the
relationship between a field/value component or between the tokens of the
same field/value componenet?

e.g. for a query like this:

field1:abc  field2:xyz

does the operator   connect field1:abc and field2:xyz , or it connects  the
tokens from "abc" and "xyz" for their respective field?


thanks,

-Hui




On 11/4/07, Yonik Seeley <yo...@apache.org> wrote:
>
> On 11/4/07, Yu-Hui Jin <yu...@gmail.com> wrote:
> > Let's say we defined a customer filed type that when querying and
> indexing,
> > the solr.LowerCaseFilterFactory is used as the last filter to low-case
> all
> > letters.  In the Analysis UI, we found tokenization is working
> correctly.
> >
> > We also defined  a custom request handler which always creates a boolean
> > query that ANDs all tokens for fielded queries (we overrided the
> > "getFieldQuery" method only).
>
> First, if all you are doing is ANDing all the tokens, you can just
> change the default operator to "AND" (q.op="AND").
>
> Analysis is done during query parsing by the query parser... if you
> create your own queries, you need to do that analysis yourself.
>
> -Yonik
>



-- 
Regards,

-Hui

Re: customer request handler doesn't envok the query tokenization chain

[Yonik Seeley <yo...@apache.org>]

On 11/4/07, Yu-Hui Jin <yu...@gmail.com> wrote:
> Let's say we defined a customer filed type that when querying and indexing,
> the solr.LowerCaseFilterFactory is used as the last filter to low-case all
> letters.  In the Analysis UI, we found tokenization is working correctly.
>
> We also defined  a custom request handler which always creates a boolean
> query that ANDs all tokens for fielded queries (we overrided the
> "getFieldQuery" method only).

First, if all you are doing is ANDing all the tokens, you can just
change the default operator to "AND" (q.op="AND").

Analysis is done during query parsing by the query parser... if you
create your own queries, you need to do that analysis yourself.

-Yonik