empty strings outputting to numeric field types

[John Blythe <jo...@curvolabs.com>]

hi all.

i'm outputting our data to xml format for solr to consume. i have several
fields storing dollar values as tdouble. they don't always exist in the
outputted rows, however, at which point they throw an error and fail at
indexing because the field is seen as an empty string (the log message: str
= '').

for now i've gotten around this by skipping out of any output for that
field in those cases, but wanted to know what the best method for
circumventing this problem is in solr (if any). i'd tried a default value
to no avail.

thanks for any thoughts-

Re: empty strings outputting to numeric field types

[John Blythe <jo...@curvolabs.com>]

Ahh, makes sense. Thanks for the clarity, Hoss.

Best-

-- 
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | john@curvolabs.com
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

On Tue, Nov 15, 2016 at 5:05 PM, Chris Hostetter <ho...@fucit.org>
wrote:

>
> : fields storing dollar values as tdouble. they don't always exist in the
> : outputted rows, however, at which point they throw an error and fail at
> : indexing because the field is seen as an empty string (the log message:
> str
> : = '').
> :
> : for now i've gotten around this by skipping out of any output for that
> : field in those cases, but wanted to know what the best method for
>
> Strictly speaking Solr isn't complaining because you give it an "empty
> string" it's complaining because you give it a string which can not be
> legally parsed as a double (or int, or float, etc...)
>
> Fixing your client to only send Solr valid numeric values, or no value
> when that's what you want for a given document, it what i would conider
> the most correct solution -- but if you want solr to ignore strings that
> aren't valid numeric values, that's what things like the
> RemoveBlankFieldUpdateProcessorFactory are for...
>
> https://lucene.apache.org/solr/6_3_0/solr-core/org/
> apache/solr/update/processor/RemoveBlankFieldUpdateProcessorFactory.html
>
> you can configure things like TrimFieldUpdateProcessorFactory and
> RegexReplaceProcessorFactory to pre-process string values to ignore
> whitespace or non decimal characters, etc... before they make it to the
> RemoveBlankFieldUpdateProcessorFactory.
>
>
>
> -Hoss
> http://www.lucidworks.com/
>

Re: empty strings outputting to numeric field types

[Chris Hostetter <ho...@fucit.org>]

: fields storing dollar values as tdouble. they don't always exist in the
: outputted rows, however, at which point they throw an error and fail at
: indexing because the field is seen as an empty string (the log message: str
: = '').
: 
: for now i've gotten around this by skipping out of any output for that
: field in those cases, but wanted to know what the best method for

Strictly speaking Solr isn't complaining because you give it an "empty 
string" it's complaining because you give it a string which can not be 
legally parsed as a double (or int, or float, etc...)

Fixing your client to only send Solr valid numeric values, or no value 
when that's what you want for a given document, it what i would conider 
the most correct solution -- but if you want solr to ignore strings that 
aren't valid numeric values, that's what things like the 
RemoveBlankFieldUpdateProcessorFactory are for...

https://lucene.apache.org/solr/6_3_0/solr-core/org/apache/solr/update/processor/RemoveBlankFieldUpdateProcessorFactory.html

you can configure things like TrimFieldUpdateProcessorFactory and 
RegexReplaceProcessorFactory to pre-process string values to ignore 
whitespace or non decimal characters, etc... before they make it to the 
RemoveBlankFieldUpdateProcessorFactory.



-Hoss
http://www.lucidworks.com/