NPE during distributed search

[Dmitry Kan <so...@gmail.com>]

Solr 4.3.1
container: jetty 9 (jetty-distribution-9.0.4.v20130625)
shard sizes: between 10G and 15G
two cores per shard, non SolrCloud mode

We have frontend solr and several shards. When searching in smaller amount
of shards, the query runs ok. When asking for larger amount of shards, the
query fails with NPE. Looking into the corresponding code, we see score
comparison:

class: ShardDoc.java
method: static Comparator comparatorScore(final String fieldName)
Code:        final float f1 = e1.score;

Looks like e1 is null. What could be the reason? Is it at all possible to
remove scoring altogether (because we don't need that)?

What else should we look into?

NPE stack trace:

ERROR org.apache.solr.servlet.SolrDispatchFilter  -
null:java.lang.NullPointerException
        at
org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardDoc.java:234)
        at
org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardDoc.java:159)
        at
org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardDoc.java:101)
        at
org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:231)
        at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
        at
org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
        at
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:863)
        at
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:625)
        at
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
        at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
        at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1820)
        at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
        at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1486)
        at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
        at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
        at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
        at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
        at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1094)
        at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:432)
        at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
        at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1028)
        at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
        at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
        at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
        at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:317)
        at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:445)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:267)
        at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:224)
        at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:601)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:532)
        at java.lang.Thread.run(Thread.java:722)