You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jérôme Etévé <je...@gmail.com> on 2008/10/01 14:42:20 UTC
Re: Discarding undefined fields in query
Hi,
yes I've got the stack trace giving me the beginning of an explanation.
One of the QueryParsers I use in my Query parser plugin is a
multifiedQueryParser and it needs a fields aware Analyzer, which I get
from the schema like this:
req.getSchema().getQueryAnalyzer();
I think it's in this analyzer that the undefined field error happens
(because for instance the field 'foo' doesn't exists in the schema,
and so it's impossible to find a specific analyzer to this field in
the schema).
The strange thing is that any QueryParser (Lucene API) is supposed to
raise a ParseException if anything wrong happens with the parsing with
the parse(String) method.
But here, it seems that the Analyzer from the schema (the one we get
from getQueryAnalyzer()) is creating it's own error ( the undefined
field one, instance of SolrException) and instead of propagating it to
the QueryParser which could have a chance to propagate it as a
standard ParseException, it seems it stops solr processing the query
directly.
Here's the full stack (with the undefined field being 'hwss' )
org.apache.solr.common.SolrException: undefined field hwss
at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1053)
at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSchema.java:373)
at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.tokenStream(IndexSchema.java:348)
at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:473)
at org.apache.lucene.queryParser.MultiFieldQueryParser.getFieldQuery(MultiFieldQueryParser.java:120)
at org.apache.lucene.queryParser.MultiFieldQueryParser.getFieldQuery(MultiFieldQueryParser.java:135)
at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1248)
at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1135)
at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1092)
at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1052)
at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:168)
at my.organisation.lucene.queryParser.MyLuceneQueryParser.parse(Unknown
Source)
at my.organisation.solr.search.MyQParser.parse(Unknown Source)
at org.apache.solr.search.QParser.getQuery(QParser.java:88)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:155)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
Cheers !
Jerome.
On Tue, Sep 30, 2008 at 10:34 PM, Yonik Seeley <yo...@apache.org> wrote:
> On Tue, Sep 30, 2008 at 2:42 PM, Jérôme Etévé <je...@gmail.com> wrote:
>> But still I have an error from the webapp when I try to query my
>> schema with non existing fields in my query ( like foo:bar ).
>>
>> I'm wondering if the query q is parsed in a very simple way somewhere
>> else (and independently from any customized QParserPlugin) and checked
>> against the schema.
>
> It should not be. Are you sure your QParser is being used?
> Does the error contain a stack trace that can pinpoint where it's coming from?
>
> -Yonik
>
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
jerome@eteve.net
Re: Discarding undefined fields in query
Posted by Chris Hostetter <ho...@fucit.org>.
: I'll catch that and deal with it then (Or is it bad programming ?) .
that's a psuedo-religious question -- i will only say that many people
recomend against using Exception catching to drive control flow, it's
called an "Exception" because it's suppose to be the "Exception" to the
norm ... if an large percentage of hte time these field names aren't going
to exist in your schema, you should probably be checking for the
non-existing field first.
Note that another way to approach this is to add a dynamicField that
matches "*" (there's an example commented out in the example schema) and
give it a fieldtype using a query Analyzer that does whatever you want
your special analyzer to do in the event of non-existent field (probably
just produce no tokens) ... the only downside to this approach is that you
won't get any errors if you inadvertantly index a document with an
unexpected field (unless you give this new fieldtype a custom *indexing*
analyzer that allways throws an exception)
-Hoss
Re: Discarding undefined fields in query
Posted by Jérôme Etévé <je...@gmail.com>.
On Tue, Oct 7, 2008 at 12:56 AM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : req.getSchema().getQueryAnalyzer();
> :
> : I think it's in this analyzer that the undefined field error happens
> : (because for instance the field 'foo' doesn't exists in the schema,
> : and so it's impossible to find a specific analyzer to this field in
> : the schema).
>
> Correct.
>
> : The strange thing is that any QueryParser (Lucene API) is supposed to
> : raise a ParseException if anything wrong happens with the parsing with
> : the parse(String) method.
> :
> : But here, it seems that the Analyzer from the schema (the one we get
> : from getQueryAnalyzer()) is creating it's own error ( the undefined
> : field one, instance of SolrException) and instead of propagating it to
> : the QueryParser which could have a chance to propagate it as a
> : standard ParseException, it seems it stops solr processing the query
> : directly.
>
> Solr isn't doing anything magical here -- it's just throwing a
> SolrException, which is a RuntimeExcepttion -- the Lucene
> QueryParser.parse method only throws a ParseException in th event of
> TooManyClauses, TokenMgrError, or an inner ParseException.
>
Ook, I get it now.
Runtime exceptions don't have to be checked at compile time, ( and
couldn't be here since the Analyzer could be anything throwing
anything).
I'll catch that and deal with it then (Or is it bad programming ?) .
Thanks for your help .
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
jerome@eteve.net
Re: Discarding undefined fields in query
Posted by Chris Hostetter <ho...@fucit.org>.
: req.getSchema().getQueryAnalyzer();
:
: I think it's in this analyzer that the undefined field error happens
: (because for instance the field 'foo' doesn't exists in the schema,
: and so it's impossible to find a specific analyzer to this field in
: the schema).
Correct.
: The strange thing is that any QueryParser (Lucene API) is supposed to
: raise a ParseException if anything wrong happens with the parsing with
: the parse(String) method.
:
: But here, it seems that the Analyzer from the schema (the one we get
: from getQueryAnalyzer()) is creating it's own error ( the undefined
: field one, instance of SolrException) and instead of propagating it to
: the QueryParser which could have a chance to propagate it as a
: standard ParseException, it seems it stops solr processing the query
: directly.
Solr isn't doing anything magical here -- it's just throwing a
SolrException, which is a RuntimeExcepttion -- the Lucene
QueryParser.parse method only throws a ParseException in th event of
TooManyClauses, TokenMgrError, or an inner ParseException.
If you want to be able to pass an analyzer to your MyLuceneQueryParser that has some
"default" behavior on unidentified fields, i would write your own that
delegates to the one provided by Solr in teh default case...
public class TolerantAnalyzer {
final IndexSchema schema;
public TolerantAnalyzer(IndexSchema s) { schema = s; }
public TokenStream tokenStream(String fieldName, Reader reader) {
if (null == schema.getFieldNoEx(fieldName) {
// your custom behavior
} else {
return schema.getQueryAnalyzer().tokenStream(fieldName, reader);
}
}
...
}
-Hoss