python response handler treats "unschema'd" fields differently

[Brian Whitman <br...@echonest.com>]

I have a solr index where we removed a field from the schema but it still
had some documents with that field in it.
Queries using the standard response handler had no problem but the
&wt=python handler would break on any query (with fl="*" or asking for that
field directly) with:

SolrHTTPException: HTTP code=400, reason=undefined_field_oldfield

I "fixed" it by putting that field back in the schema.

One related weirdness is that fl=oldfield would cause the exception but not
fl=othernonschemafield -- that is, it would only break on field names that
were not in schema but were in the documents.

I know this is undefined behavior territory but it was still weird that the
standard response writer does not do this-- if you give a nonexistent field
name to fl on wt=standard, either one that is in documents or is not -- it
happily performs the query just skipping the ones that are not in the
schema.

Re: python response handler treats "unschema'd" fields differently

[Chris Hostetter <ho...@fucit.org>]

: Seems like we could handle this 2 ways... leave out the field if it's
: not defined in the schema, or include it and write it out as a string.
:  I think either would probably be more useful than throwing an error
: (which isn't really a request error but rather a schema/indexing
: error).

i'd leave the field out -- be consistent with the XML format.

Assuming it can be written out cleanly as a string could be very very bad 
in the worse case (i'm thinking of large encoded binary fields, or other 
types of data that could be expensive to do all the string escaping on)




-Hoss

Re: python response handler treats "unschema'd" fields differently

[Yonik Seeley <yo...@lucidimagination.com>]

Seems like we could handle this 2 ways... leave out the field if it's
not defined in the schema, or include it and write it out as a string.
 I think either would probably be more useful than throwing an error
(which isn't really a request error but rather a schema/indexing
error).

Thoughts?

-Yonik
http://www.lucidimagination.com


On Fri, Apr 17, 2009 at 4:36 PM, Brian Whitman <br...@echonest.com> wrote:
> I have a solr index where we removed a field from the schema but it still
> had some documents with that field in it.
> Queries using the standard response handler had no problem but the
> &wt=python handler would break on any query (with fl="*" or asking for that
> field directly) with:
>
> SolrHTTPException: HTTP code=400, reason=undefined_field_oldfield
>
> I "fixed" it by putting that field back in the schema.
>
> One related weirdness is that fl=oldfield would cause the exception but not
> fl=othernonschemafield -- that is, it would only break on field names that
> were not in schema but were in the documents.
>
> I know this is undefined behavior territory but it was still weird that the
> standard response writer does not do this-- if you give a nonexistent field
> name to fl on wt=standard, either one that is in documents or is not -- it
> happily performs the query just skipping the ones that are not in the
> schema.
>