You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2013/01/05 23:22:12 UTC
[jira] [Resolved] (SOLR-4275) TrieTokenizer causes StringIOOBE when
input is empty instead of returning no token
[ https://issues.apache.org/jira/browse/SOLR-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved SOLR-4275.
---------------------------------
Resolution: Fixed
Committed to 4.x and trunk.
> TrieTokenizer causes StringIOOBE when input is empty instead of returning no token
> ----------------------------------------------------------------------------------
>
> Key: SOLR-4275
> URL: https://issues.apache.org/jira/browse/SOLR-4275
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4275.patch
>
>
> When you use the admin interface and select a trie field (e.g. tint) and enter nothing into the field, the tokenizer should normally return no tokens. TrieTokenizer instead gets and SIOOBE because read() into the charbuffer returns -1 (end of stream). This is used to initialize the string's length...
> The problem is mostly affecting the analysis request handler and query parsing, but while indexing the values, Solr uses NumericField and not the tokenizer directly. The solr admin UI has the additional problem that you get a strange exception if you fill in the number on the left, but leave the query (right empty).
> The fix is to modify the tokenizer to behave like a real tokenizer:
> - correct the read loop to look like the one from KeywordTokenizer. The current loop is not guaranteed to work with unbuffered readers (Solr always uses StringReaders so this is no issue, but who knows)
> - if the resulting string is empty (total len == 0), set a boolean to false and make the incrementToken/close/end methods not delegate and return false.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org