You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by coolpriya5 <co...@gmail.com> on 2013/04/11 00:19:38 UTC
NPE when faceting TEXTfield in a distributed search query
Hi,
I have a nullable TextField with field type as follows(field
name=fun_group):-
<fieldType name="lowercase_sort_missing_first" class="solr.TextField"
sortMissingFirst="true" positionIncrementGap="100">
<analyzer><tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/></analyzer>
</fieldType>
field definition is:-
<field name="fun_group" type="lowercase_sort_missing_first" indexed="true"
stored="true" multiValued="false"/>
I am running a distributed search(using shards= param) across 13 shards
while faceting on fun_group with the following params:-
http://<shard1>/solr/mycore/select?facet=true&facet.field=fun_group&facet.mincount=1&rows=0&facet.limit=10000&facet.offset=0&q=(((my_id:4024
AND ((flag:1 OR flag:0)))) AND
class_id:2604)&shards=<shard1>/solr/mycore,<shard2>/solr/mycore,<shard3>/solr/mycore,<shard4>/solr/mycore,<shard5>/solr/mycore,<shard6>/solr/mycore,<shard7>/solr/mycore,<shard8>/solr/mycore,<shard9>/solr/mycore,<shard10>/solr/mycore,<shard11>/solr/mycore,<shard12>/solr/mycore,<shard13>/solr/mycore
I get Internal Server Error on shard6. In the logs, I see the NPE from
org.apache.solr.servlet.SolrDispatchFilter.
If I change the facet.limit from 10000 to 5000 keeping the
facet.offset=0,there are no exceptions. Now again if try facet.limit=5000
and facet.offset=5000 , I get the same exception. I tried facet.limit=4000
and facet.offset=5000, it worked fine. Please help
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: NPE when faceting TEXTfield in a distributed search query
Posted by Michael Ryan <mr...@moreover.com>.
Yes, this is a distributed search thing. In a distributed search, it will first make a somewhat normal facet request to all of the shards, get back the facet values, then make a second request in order to get the full counts of the facet values - this second request contains a list of facet terms that were not returned in the first response but were found on the other shards. Though, if only one shard even has this facet data, I would kind of expect you to not see this...
If Tomcat is truncating the request, the NPE could be due to https://issues.apache.org/jira/browse/SOLR-435.
Are you able to see the query in the shard 6 log for the working facet.limit=5000&facet.offset=0 query?
-Michael
-----Original Message-----
From: coolpriya5 [mailto:coolpriya5@gmail.com]
Sent: Wednesday, April 10, 2013 9:16 PM
To: solr-user@lucene.apache.org
Subject: RE: NPE when faceting TEXTfield in a distributed search query
I'm using tomcat. Also in such a case, why wouldn't the same error occur when I run the same query on shard 6 alone? Is this a limitation of distributed search?
Shard 6 is the only shard that has data for this query.
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055208.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: NPE when faceting TEXTfield in a distributed search query
Posted by coolpriya5 <co...@gmail.com>.
I'm using tomcat. Also in such a case, why wouldn't the same error occur when
I run the same query on shard 6 alone? Is this a limitation of distributed
search?
Shard 6 is the only shard that has data for this query.
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055208.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: NPE when faceting TEXTfield in a distributed search query
Posted by Michael Ryan <mr...@moreover.com>.
Large facet.limit values cause a very large amount of form data to be sent to the shards, though I'm not sure why this would cause a NullPointerException. Perhaps the web server you are using is truncating the data instead of returning a form too large error, which is somehow causing an NPE. Are you using Jetty, Tomcat, or something else?
-Michael
-----Original Message-----
From: coolpriya5 [mailto:coolpriya5@gmail.com]
Sent: Wednesday, April 10, 2013 7:17 PM
To: solr-user@lucene.apache.org
Subject: Re: NPE when faceting TEXTfield in a distributed search query
Solr Version is 3.4. As for stacktrace, I tried setting logger level to FINEST on the solr admin logging page and it still doesn't print the stacktrace. All I get are one liners:-
2013-04-10 17:09:59,889 [http-xxxx-18] ERROR [Marker: ]
org.apache.solr.core.SolrCore :
java.lang.NullPointerException
2013-04-10 17:09:59,890 [http-xxxx-18] INFO [Marker: ]
org.apache.solr.core.SolrCore : [catalog] webapp=/solr
path=/select params={} status=500 QTime=1
2013-04-10 17:09:59,890 [http-xxx-18] ERROR [Marker: ]
org.apache.solr.servlet.SolrDispatchFilter :
java.lang.NullPointerException
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055189.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Posted by coolpriya5 <co...@gmail.com>.
Solr Version is 3.4. As for stacktrace, I tried setting logger level to
FINEST on the solr admin logging page and it still doesn't print the
stacktrace. All I get are one liners:-
2013-04-10 17:09:59,889 [http-xxxx-18] ERROR [Marker: ]
org.apache.solr.core.SolrCore :
java.lang.NullPointerException
2013-04-10 17:09:59,890 [http-xxxx-18] INFO [Marker: ]
org.apache.solr.core.SolrCore : [catalog] webapp=/solr
path=/select params={} status=500 QTime=1
2013-04-10 17:09:59,890 [http-xxx-18] ERROR [Marker: ]
org.apache.solr.servlet.SolrDispatchFilter :
java.lang.NullPointerException
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055189.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Posted by Jack Krupansky <ja...@basetechnology.com>.
If the NPE was in SolrDispatchFilter, it could relate to some limit on the
HTTP request or response size.
Again, we need the full stack trace, and the Solr release.
-- Jack Krupansky
-----Original Message-----
From: coolpriya5
Sent: Wednesday, April 10, 2013 7:06 PM
To: solr-user@lucene.apache.org
Subject: Re: NPE when faceting TEXTfield in a distributed search query
Number of values fun_group on shard 6 = 48000
Max length of fun_group is 20 chars
If I run the facet on just shard6 it doesn't error out no matter the
facet.limit. Also this query returns results only from shard 6 since the
my_id:4024 belongs to shard 6.
--
View this message in context:
http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055185.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Posted by coolpriya5 <co...@gmail.com>.
Number of values fun_group on shard 6 = 48000
Max length of fun_group is 20 chars
If I run the facet on just shard6 it doesn't error out no matter the
facet.limit. Also this query returns results only from shard 6 since the
my_id:4024 belongs to shard 6.
--
View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055185.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Posted by Jack Krupansky <ja...@basetechnology.com>.
Ah... I see now that you are using the keyword tokenizer that should
preserve the phrase structure of the text.
You haven't detailed the exception stack trace.
What are the numbers in terms of number of values and average length of each
value?
-- Jack Krupansky
-----Original Message-----
From: Jack Krupansky
Sent: Wednesday, April 10, 2013 6:30 PM
To: solr-user@lucene.apache.org
Subject: Re: NPE when faceting TEXTfield in a distributed search query
Are you sure you want to facet on a text field??? That will facet on the
individual terms, which isn't usually very useful.
Usually, people want to facet on full phrases or entire strings, so they do
a schema copyField from the text field to a string field and then facet on
the string field.
-- Jack Krupansky
-----Original Message-----
From: coolpriya5
Sent: Wednesday, April 10, 2013 6:19 PM
To: solr-user@lucene.apache.org
Subject: NPE when faceting TEXTfield in a distributed search query
Hi,
I have a nullable TextField with field type as follows(field
name=fun_group):-
<fieldType name="lowercase_sort_missing_first" class="solr.TextField"
sortMissingFirst="true" positionIncrementGap="100">
<analyzer><tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/></analyzer>
</fieldType>
field definition is:-
<field name="fun_group" type="lowercase_sort_missing_first" indexed="true"
stored="true" multiValued="false"/>
I am running a distributed search(using shards= param) across 13 shards
while faceting on fun_group with the following params:-
http://<shard1>/solr/mycore/select?facet=true&facet.field=fun_group&facet.mincount=1&rows=0&facet.limit=10000&facet.offset=0&q=(((my_id:4024
AND ((flag:1 OR flag:0)))) AND
class_id:2604)&shards=<shard1>/solr/mycore,<shard2>/solr/mycore,<shard3>/solr/mycore,<shard4>/solr/mycore,<shard5>/solr/mycore,<shard6>/solr/mycore,<shard7>/solr/mycore,<shard8>/solr/mycore,<shard9>/solr/mycore,<shard10>/solr/mycore,<shard11>/solr/mycore,<shard12>/solr/mycore,<shard13>/solr/mycore
I get Internal Server Error on shard6. In the logs, I see the NPE from
org.apache.solr.servlet.SolrDispatchFilter.
If I change the facet.limit from 10000 to 5000 keeping the
facet.offset=0,there are no exceptions. Now again if try facet.limit=5000
and facet.offset=5000 , I get the same exception. I tried facet.limit=4000
and facet.offset=5000, it worked fine. Please help
--
View this message in context:
http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Posted by Jack Krupansky <ja...@basetechnology.com>.
Are you sure you want to facet on a text field??? That will facet on the
individual terms, which isn't usually very useful.
Usually, people want to facet on full phrases or entire strings, so they do
a schema copyField from the text field to a string field and then facet on
the string field.
-- Jack Krupansky
-----Original Message-----
From: coolpriya5
Sent: Wednesday, April 10, 2013 6:19 PM
To: solr-user@lucene.apache.org
Subject: NPE when faceting TEXTfield in a distributed search query
Hi,
I have a nullable TextField with field type as follows(field
name=fun_group):-
<fieldType name="lowercase_sort_missing_first" class="solr.TextField"
sortMissingFirst="true" positionIncrementGap="100">
<analyzer><tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/></analyzer>
</fieldType>
field definition is:-
<field name="fun_group" type="lowercase_sort_missing_first" indexed="true"
stored="true" multiValued="false"/>
I am running a distributed search(using shards= param) across 13 shards
while faceting on fun_group with the following params:-
http://<shard1>/solr/mycore/select?facet=true&facet.field=fun_group&facet.mincount=1&rows=0&facet.limit=10000&facet.offset=0&q=(((my_id:4024
AND ((flag:1 OR flag:0)))) AND
class_id:2604)&shards=<shard1>/solr/mycore,<shard2>/solr/mycore,<shard3>/solr/mycore,<shard4>/solr/mycore,<shard5>/solr/mycore,<shard6>/solr/mycore,<shard7>/solr/mycore,<shard8>/solr/mycore,<shard9>/solr/mycore,<shard10>/solr/mycore,<shard11>/solr/mycore,<shard12>/solr/mycore,<shard13>/solr/mycore
I get Internal Server Error on shard6. In the logs, I see the NPE from
org.apache.solr.servlet.SolrDispatchFilter.
If I change the facet.limit from 10000 to 5000 keeping the
facet.offset=0,there are no exceptions. Now again if try facet.limit=5000
and facet.offset=5000 , I get the same exception. I tried facet.limit=4000
and facet.offset=5000, it worked fine. Please help
--
View this message in context:
http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html
Sent from the Solr - User mailing list archive at Nabble.com.