You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Matthew Nigl <ma...@gmail.com> on 2014/10/08 14:47:23 UTC
NullPointerException for ExternalFileField when key field has no terms
Hi,
I use various ID fields as the keys for various ExternalFileField fields,
and I have noticed that I will sometimes get the following error:
ERROR org.apache.solr.servlet.SolrDispatchFilter û
null:java.lang.NullPointerException
at
org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273)
at
org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51)
at
org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147)
at
org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190)
at
org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141)
at
org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84)
at
org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95)
at
org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252)
at
org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170)
at
org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184)
at
org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300)
at
org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96)
at
org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61)
at
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
The source code referenced in the error is below (FileFloatSource.java:273):
TermsEnum termsEnum = MultiFields.getTerms(reader, idName).iterator(null);
So if there are no terms in the index for the key field, then getTerms will
return null, and of course trying to call iterator on null will cause the
exception.
For my use-case, it makes sense that the key field may have no terms
(initially) because there are various types of documents sharing the index,
and they will not all exist at the onset. The default value for the EFF
would suffice in those cases.
Is this worthy of a JIRA? I have gone through whatever documentation I can
find for ExternalFileField and I can't seem to find anything about
requiring key terms first. It seems that this error is not encountered
often because users generally set the unique key field as the external file
key field, so it always exists.
The workaround is to ensure at least one document contains a value for the
key fields.
Regards,
Matt
Re: NullPointerException for ExternalFileField when key field has no terms
Posted by Matthew Nigl <ma...@gmail.com>.
Thanks Markus. I initially interpreted the line "It's OK to have a keyField
value that can't be found in the index" as meaning that the key field value
in the external file does not have to exist as a term in the index.
On 8 October 2014 23:56, Markus Jelsma <ma...@openindex.io> wrote:
> Hi - yes it is worth a ticket as the javadoc says it is ok:
>
> http://lucene.apache.org/solr/4_10_1/solr-core/org/apache/solr/schema/ExternalFileField.html
>
>
> -----Original message-----
> > From:Matthew Nigl <ma...@gmail.com>
> > Sent: Wednesday 8th October 2014 14:48
> > To: solr-user@lucene.apache.org
> > Subject: NullPointerException for ExternalFileField when key field has
> no terms
> >
> > Hi,
> >
> > I use various ID fields as the keys for various ExternalFileField fields,
> > and I have noticed that I will sometimes get the following error:
> >
> > ERROR org.apache.solr.servlet.SolrDispatchFilter û
> > null:java.lang.NullPointerException
> > at
> >
> org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273)
> > at
> >
> org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51)
> > at
> >
> org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147)
> > at
> >
> org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190)
> > at
> >
> org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141)
> > at
> >
> org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84)
> > at
> >
> org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95)
> > at
> >
> org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252)
> > at
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170)
> > at
> >
> org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184)
> > at
> >
> org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300)
> > at
> >
> org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96)
> > at
> >
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> > at org.eclipse.jetty.server.Server.handle(Server.java:368)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> > at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> > at
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> > at
> > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> > at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> > at
> >
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> > at java.lang.Thread.run(Unknown Source)
> >
> >
> >
> > The source code referenced in the error is below
> (FileFloatSource.java:273):
> >
> > TermsEnum termsEnum = MultiFields.getTerms(reader,
> idName).iterator(null);
> >
> > So if there are no terms in the index for the key field, then getTerms
> will
> > return null, and of course trying to call iterator on null will cause the
> > exception.
> >
> > For my use-case, it makes sense that the key field may have no terms
> > (initially) because there are various types of documents sharing the
> index,
> > and they will not all exist at the onset. The default value for the EFF
> > would suffice in those cases.
> >
> > Is this worthy of a JIRA? I have gone through whatever documentation I
> can
> > find for ExternalFileField and I can't seem to find anything about
> > requiring key terms first. It seems that this error is not encountered
> > often because users generally set the unique key field as the external
> file
> > key field, so it always exists.
> >
> > The workaround is to ensure at least one document contains a value for
> the
> > key fields.
> >
> > Regards,
> > Matt
> >
>
RE: NullPointerException for ExternalFileField when key field has no
terms
Posted by Markus Jelsma <ma...@openindex.io>.
Hi - yes it is worth a ticket as the javadoc says it is ok:
http://lucene.apache.org/solr/4_10_1/solr-core/org/apache/solr/schema/ExternalFileField.html
-----Original message-----
> From:Matthew Nigl <ma...@gmail.com>
> Sent: Wednesday 8th October 2014 14:48
> To: solr-user@lucene.apache.org
> Subject: NullPointerException for ExternalFileField when key field has no terms
>
> Hi,
>
> I use various ID fields as the keys for various ExternalFileField fields,
> and I have noticed that I will sometimes get the following error:
>
> ERROR org.apache.solr.servlet.SolrDispatchFilter û
> null:java.lang.NullPointerException
> at
> org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273)
> at
> org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51)
> at
> org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147)
> at
> org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190)
> at
> org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141)
> at
> org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84)
> at
> org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95)
> at
> org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252)
> at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170)
> at
> org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184)
> at
> org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300)
> at
> org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96)
> at
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61)
> at
> org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Unknown Source)
>
>
>
> The source code referenced in the error is below (FileFloatSource.java:273):
>
> TermsEnum termsEnum = MultiFields.getTerms(reader, idName).iterator(null);
>
> So if there are no terms in the index for the key field, then getTerms will
> return null, and of course trying to call iterator on null will cause the
> exception.
>
> For my use-case, it makes sense that the key field may have no terms
> (initially) because there are various types of documents sharing the index,
> and they will not all exist at the onset. The default value for the EFF
> would suffice in those cases.
>
> Is this worthy of a JIRA? I have gone through whatever documentation I can
> find for ExternalFileField and I can't seem to find anything about
> requiring key terms first. It seems that this error is not encountered
> often because users generally set the unique key field as the external file
> key field, so it always exists.
>
> The workaround is to ensure at least one document contains a value for the
> key fields.
>
> Regards,
> Matt
>