Set result limit to MAX_VALUE causes NullPointerException

[Jörg Kiegeland <ki...@ikv.de>]

Hi,

if I do a query and set the result limit to Integer.MAX_VALUE, it causes 
following exception:

org.apache.solr.client.solrj.SolrServerException: Error executing query
    at 
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:86)
    at 
org.apache.solr.client.solrj.impl.BaseSolrServer.query(BaseSolrServer.java:99)
...
Caused by: org.apache.solr.common.SolrException: 
null__javalangNullPointerException___at_orgapachesolrutilSolrPluginUtilsoptimizePreFetchDocsSolrPluginUtilsjava253___at_orgapachesolrhandlerStandardRequestHandlerhandleRequestBodyStandardRequestHandlerjava104___at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava117___at_orgapachesolrcoreSolrCoreexecuteSolrCorejava811___at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava206___at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava174___at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089___at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365___at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181___at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712___at_orgmortbayjettyhandlerHandlerWrapperhandleHandlerWrapperjava139___at_orgmortbayjettyServerhandleServerjava285___at_orgmortbayjettyHttpConnectionhandleRequestHttpConnectionjava502___at_orgmortbayjettyHttpConnection$RequestHandlerheaderCompleteHttpConnectionjava821___at_orgmortbayjettyHttpParserparseNextHttpParserjava513___at_orgmortbayjettyHttpParserparseAvailableHttpParserjava208___at_orgmortbayjettyHttpConnectionhandleHttpConnectionjava378___at_orgmortbayjettybioSocketConnector$ConnectionrunSocketConnectorjava226___at_orgmortbaythreadBoundedThreadPool$PoolThreadrunBoundedThreadPooljava442__

null__javalangNullPointerException___at_orgapachesolrutilSolrPluginUtilsoptimizePreFetchDocsSolrPluginUtilsjava253___at_orgapachesolrhandlerStandardRequestHandlerhandleRequestBodyStandardRequestHandlerjava104___at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava117___at_orgapachesolrcoreSolrCoreexecuteSolrCorejava811___at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava206___at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava174___at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089___at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365___at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181___at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712___at_orgmortbayjettyhandlerHandlerWrapperhandleHandlerWrapperjava139___at_orgmortbayjettyServerhandleServerjava285___at_orgmortbayjettyHttpConnectionhandleRequestHttpConnectionjava502___at_orgmortbayjettyHttpConnection$RequestHandlerheaderCompleteHttpConnectionjava821___at_orgmortbayjettyHttpParserparseNextHttpParserjava513___at_orgmortbayjettyHttpParserparseAvailableHttpParserjava208___at_orgmortbayjettyHttpConnectionhandleHttpConnectionjava378___at_orgmortbayjettybioSocketConnector$ConnectionrunSocketConnectorjava226___at_orgmortbaythreadBoundedThreadPool$PoolThreadrunBoundedThreadPooljava442__

request: 
http://localhost:8984/example/select?q=(dim_features:"_JrsckpOREdy_bZ3jDobN3g" 
OR dim_features:"_JrsckJOREdy_bZ3jDobN3g" OR 
dim_features:"_JrsckZOREdy_bZ3jDobN3g") AND NOT 
dim_features:"_Jrsck5OREdy_bZ3jDobN3g" AND to_revision:[2147483647 TO 
2147483647] AND NOT from_revision:{2147483647 TO 
2147483647}&rows=2147483647&wt=xml&version=2.2
    at 
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:230)
    at 
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:80)
    ... 29 more






This is Solr 1.3 using HTTP. I used to get a NullPointerException also 
in Solr 1.2 using the embedded-API.


I now set a high value like 10000 as result limit, which is always 
enough for my needs, but I nevertheless wanted to point at this error.

Greetings
Jörg

Re: Set result limit to MAX_VALUE causes NullPointerException

[Chris Hostetter <ho...@fucit.org>]

: I now set a high value like 10000 as result limit, which is always enough for
: my needs, but I nevertheless wanted to point at this error.

For the record: while Yonik has fixed solr so that it *can* accept 
MAX_VALUE, doesn't mean you *should* use MAX_VALUE.

even if you want *all* the results, no matter ow many there are, if you 
are confident you know what the upper bound is (ie: 10000 "is always 
enough" for your needs) you should use that upper bound and put an 
assertion in your client that checks hte value of numFound and reports an 
error if it's higher then your epectation.

Example: if you have 1 million documents, and you *know* a query is going 
to match no more then 1000 and you want them all then say rows=1000 
instead of rows='MAX_VALUE' ...

1) it might allow Solr to be more be more efficient in processing your 
request since it knows you only want a certain number anyway. 9ie: in 
allocating arrays and such)

2) it protects you in the event that you are wrong.

what if because of a bug in your indexing code, or some action taken by 
someone else, it turns out that your query actually matches all 1 million 
docs in your index ... by asking for "all" of them, you 
not only risk crashing your client when it gets back 10,000 times 
as much data as it exepcted but you could also hoze your network, and 
maybe even crash the Solr server itself.






-Hoss

Re: Set result limit to MAX_VALUE causes NullPointerException

[Yonik Seeley <yo...@apache.org>]

On Nov 15, 2007 10:49 AM, Jörg Kiegeland <ki...@ikv.de> wrote:
> if I do a query and set the result limit to Integer.MAX_VALUE, it causes
> following exception:

Thanks for the bug report.  Could you try the patch in
https://issues.apache.org/jira/browse/SOLR-413

-Yonik