You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ken Krugler <kk...@transpac.com> on 2012/01/13 22:04:04 UTC
JSON & XML response writer issues with short & binary fields
I finally got around to looking at why short field values are returned as "java.lang.Short:<value>".
Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used:
// default... for debugging only
writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
The same thing happens when you have a binary field, since val in that case is byte[], so you get "[B:[B@<address of byte array>"
Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema.
This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere.
-- Ken
PS - any good reason why XMLWriter is a final class? I created my own fixed version of JSONResponseWriter w/o much effort because I could subclass it, but XMLWriter being final makes it hard (impossible?) to do the same, since there are numerous internal methods that take an explicit XMLWriter object as a parameter.
--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Re: JSON & XML response writer issues with short & binary fields
Posted by Ken Krugler <kk...@transpac.com>.
On Jan 13, 2012, at 1:39pm, Yonik Seeley wrote:
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Fri, Jan 13, 2012 at 4:22 PM, Yonik Seeley
> <yo...@lucidimagination.com> wrote:
>> On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler
>> <kk...@transpac.com> wrote:
>>> I finally got around to looking at why short field values are returned as "java.lang.Short:<value>".
>>>
>>> Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used:
>>>
>>> // default... for debugging only
>>> writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
>>>
>>> The same thing happens when you have a binary field, since val in that case is byte[], so you get "[B:[B@<address of byte array>"
>>>
>>> Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema.
>>>
>>> This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere.
>>
>> Hmmm, Ryan recently restructured some of the writer code to support
>> the pseudo-field feature. A quick look at the code seems like
>> FieldType.write() methods are not used anymore (the Document is
>> transformed into a SolrDocument and writeVal is used for each value).
>
> Double hmmm... I see this in writeVal()
>
> } else if (val instanceof IndexableField) {
> IndexableField f = (IndexableField)val;
> SchemaField sf = schema.getFieldOrNull( f.name() );
> if( sf != null ) {
> sf.getType().write(this, name, f);
> }
>
> So my initial quick analysis of FieldType.write() not being used
> anymore doesn't look correct.
> Anyway, please do open an issue and we'll get to the bottom of it.
Thanks for the fast response - I was beginning to worry that nobody read my posts :)
See https://issues.apache.org/jira/browse/SOLR-3035
I've attached some test code to the issue, plus a simple fix for the JSON case.
Regards,
-- Ken
--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Re: JSON & XML response writer issues with short & binary fields
Posted by Yonik Seeley <yo...@lucidimagination.com>.
-Yonik
http://www.lucidimagination.com
On Fri, Jan 13, 2012 at 4:22 PM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler
> <kk...@transpac.com> wrote:
>> I finally got around to looking at why short field values are returned as "java.lang.Short:<value>".
>>
>> Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used:
>>
>> // default... for debugging only
>> writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
>>
>> The same thing happens when you have a binary field, since val in that case is byte[], so you get "[B:[B@<address of byte array>"
>>
>> Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema.
>>
>> This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere.
>
> Hmmm, Ryan recently restructured some of the writer code to support
> the pseudo-field feature. A quick look at the code seems like
> FieldType.write() methods are not used anymore (the Document is
> transformed into a SolrDocument and writeVal is used for each value).
Double hmmm... I see this in writeVal()
} else if (val instanceof IndexableField) {
IndexableField f = (IndexableField)val;
SchemaField sf = schema.getFieldOrNull( f.name() );
if( sf != null ) {
sf.getType().write(this, name, f);
}
So my initial quick analysis of FieldType.write() not being used
anymore doesn't look correct.
Anyway, please do open an issue and we'll get to the bottom of it.
-Yonik
http://www.lucidimagination.com
Re: JSON & XML response writer issues with short & binary fields
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler
<kk...@transpac.com> wrote:
> I finally got around to looking at why short field values are returned as "java.lang.Short:<value>".
>
> Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used:
>
> // default... for debugging only
> writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
>
> The same thing happens when you have a binary field, since val in that case is byte[], so you get "[B:[B@<address of byte array>"
>
> Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema.
>
> This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere.
Hmmm, Ryan recently restructured some of the writer code to support
the pseudo-field feature. A quick look at the code seems like
FieldType.write() methods are not used anymore (the Document is
transformed into a SolrDocument and writeVal is used for each value).
Guess this highlights deficiencies in our tests as well surrounding
short, byte and binary.
Could you open an issue for this?
-Yonik
http://www.lucidimagination.com