You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Bram Van Dam <br...@intix.eu> on 2016/11/09 09:33:04 UTC
5.5.3: fieldValueCache auto-warming error
Hey folks,
I'm frequently getting the following error, which has me a little puzzled:
Error during auto-warming of
key:text:org.apache.solr.common.SolrException:
java.lang.IllegalStateException: Too many values for UnInvertedField
faceting on field text
This is strange because the field in question is a full text field,
which will never ever be used in faceting. I don't understand why Solr
is trying to build an uninverted index for faceting on this field during
auto-warming.
Should I turn off the auto-warming of fieldValueCache? But then that
will probably affect overall performance of other fields, which is
something I'd like to avoid.
Is there any way to exclude this field from being cached in the manner?
Thanks a bunch. Stack trace and field definitions below.
- Bram
Field:
<fieldType name="text" class="solr.TextField" omitNorms="true"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<field name="text" type="text" docValues="false" indexed="true"
stored="false" required="false" multiValued="false" omitNorms="true"
termVectors="false" termPositions="false" termOffsets="false" />
Cache definition:
<fieldValueCache class="solr.FastLRUCache" size="10000"
autowarmCount="1024" showItems="32" />
Full stack trace:
2016-11-08
03:35:57.196/UTC|ERROR|searcherExecutor-27-thread-1-processing-x:myIndex||||x:myIndex|org.apache.solr.search.FastLRUCache|Error
during auto-warming of key:text:org.apache.solr.common.SolrException:
java.lang.IllegalStateException: Too many values for UnInvertedField
faceting on field text
at
org.apache.solr.search.facet.UnInvertedField.<init>(UnInvertedField.java:194)
at
org.apache.solr.search.facet.UnInvertedField.getUnInvertedField(UnInvertedField.java:595)
at
org.apache.solr.search.SolrIndexSearcher$1.regenerateItem(SolrIndexSearcher.java:523)
at org.apache.solr.search.FastLRUCache.warm(FastLRUCache.java:163)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:2320)
at org.apache.solr.core.SolrCore$4.call(SolrCore.java:1851)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Too many values for
UnInvertedField faceting on field text
at org.apache.lucene.uninverting.DocTermOrds.uninvert(DocTermOrds.java:489)
at
org.apache.solr.search.facet.UnInvertedField.<init>(UnInvertedField.java:192)
... 10 more
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Erick Erickson <er...@gmail.com>.
Thanks for letting us know. I raised a JIRA but I won't have time to
work on it in the foreseeable future.
Erick
On Tue, Nov 15, 2016 at 6:24 AM, Bram Van Dam <br...@intix.eu> wrote:
> On 11/11/16 18:08, Bram Van Dam wrote:
>> On 10/11/16 17:10, Erick Erickson wrote:
>>> Just facet on the text field yourself ;)....
>
> Quick update: you were right. One of the users managed to find a bug in
> our application which enabled them to facet on the text field. It would
> be still be nice if Solr wouldn't try to keep caching a broken query (or
> an impossible facet field), but we can work around the issue by fixing
> our own bug.
>
> Thanks!
>
> - Bram
>
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Bram Van Dam <br...@intix.eu>.
On 11/11/16 18:08, Bram Van Dam wrote:
> On 10/11/16 17:10, Erick Erickson wrote:
>> Just facet on the text field yourself ;)....
Quick update: you were right. One of the users managed to find a bug in
our application which enabled them to facet on the text field. It would
be still be nice if Solr wouldn't try to keep caching a broken query (or
an impossible facet field), but we can work around the issue by fixing
our own bug.
Thanks!
- Bram
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Bram Van Dam <br...@intix.eu>.
On 10/11/16 17:10, Erick Erickson wrote:
> Just facet on the text field yourself ;)....
Wish I could, this is on premise over at a client, access is difficult
and their response time is pretty bad on public holidays and weekends.
So I'm basically twiddling my thumbs while waiting to get more log files
:-) I haven't been able to reproduce the problem locally, but there
could be any number of contributing factors that I'm missing.
> Kidding aside, this should be in the clear from the logs, my guess is
> that the first time you see an OOM error in the logs the query will be
> in the file also.
We generally prefer "fail hard fast", so I think we are running with the
OOM killer script in most environments. I don't think they've gone OOM
in this case, though something else could have gone wrong undected.
I hope I'll know more after the weekend.
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Erick Erickson <er...@gmail.com>.
Just facet on the text field yourself ;)....
Kidding aside, this should be in the clear from the logs, my guess is
that the first time you see an OOM error in the logs the query will be
in the file also.
You could also fire up Solr with the OOM killer script (see solr/bin).
Kind of harsh, but you'd know one of these came through because the
Solr instance would go down.
Best,
Erick
On Thu, Nov 10, 2016 at 3:37 AM, Bram Van Dam <br...@intix.eu> wrote:
> On 09/11/16 16:59, Erick Erickson wrote:
>> But my bet is that you _are_ doing something that uninverts the text
>> field (obviously inadvertently). If you restart Solr and monitor the
>> log until the first time you see this exception, what do the queries
>> show? My guess is that once you get some query in your
>> queryResultCache or filterCache it gets recycled and produces this on
>> autowarm rather than the fieldValueCache, but that's a total guess.
>
> Good point. I'll try to log the queries and see if anything comes up.
> Though if it was a user-initiated error it might be impossible to get
> them to reproduce it.
>
> I'll keep you updated.
>
> Thanks!
>
> - Bram
>
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Bram Van Dam <br...@intix.eu>.
On 09/11/16 16:59, Erick Erickson wrote:
> But my bet is that you _are_ doing something that uninverts the text
> field (obviously inadvertently). If you restart Solr and monitor the
> log until the first time you see this exception, what do the queries
> show? My guess is that once you get some query in your
> queryResultCache or filterCache it gets recycled and produces this on
> autowarm rather than the fieldValueCache, but that's a total guess.
Good point. I'll try to log the queries and see if anything comes up.
Though if it was a user-initiated error it might be impossible to get
them to reproduce it.
I'll keep you updated.
Thanks!
- Bram
Re: 5.5.3: fieldValueCache auto-warming error
Posted by Erick Erickson <er...@gmail.com>.
That is strange. If you turn off autowarming for fieldValueCache and
this goes away that would be _really_ strange indeed.
But my bet is that you _are_ doing something that uninverts the text
field (obviously inadvertently). If you restart Solr and monitor the
log until the first time you see this exception, what do the queries
show? My guess is that once you get some query in your
queryResultCache or filterCache it gets recycled and produces this on
autowarm rather than the fieldValueCache, but that's a total guess.
And my guess is correct, there's probably _is_ a JIRA here to not put
anything in caches (filter or queryResult) that's an exception......
Puzzled,
Erick
On Wed, Nov 9, 2016 at 1:33 AM, Bram Van Dam <br...@intix.eu> wrote:
> Hey folks,
>
> I'm frequently getting the following error, which has me a little puzzled:
>
> Error during auto-warming of
> key:text:org.apache.solr.common.SolrException:
> java.lang.IllegalStateException: Too many values for UnInvertedField
> faceting on field text
>
> This is strange because the field in question is a full text field,
> which will never ever be used in faceting. I don't understand why Solr
> is trying to build an uninverted index for faceting on this field during
> auto-warming.
>
> Should I turn off the auto-warming of fieldValueCache? But then that
> will probably affect overall performance of other fields, which is
> something I'd like to avoid.
>
> Is there any way to exclude this field from being cached in the manner?
>
> Thanks a bunch. Stack trace and field definitions below.
>
> - Bram
>
> Field:
>
> <fieldType name="text" class="solr.TextField" omitNorms="true"
> positionIncrementGap="100">
> <analyzer>
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
>
> <field name="text" type="text" docValues="false" indexed="true"
> stored="false" required="false" multiValued="false" omitNorms="true"
> termVectors="false" termPositions="false" termOffsets="false" />
>
> Cache definition:
>
> <fieldValueCache class="solr.FastLRUCache" size="10000"
> autowarmCount="1024" showItems="32" />
>
> Full stack trace:
>
> 2016-11-08
> 03:35:57.196/UTC|ERROR|searcherExecutor-27-thread-1-processing-x:myIndex||||x:myIndex|org.apache.solr.search.FastLRUCache|Error
> during auto-warming of key:text:org.apache.solr.common.SolrException:
> java.lang.IllegalStateException: Too many values for UnInvertedField
> faceting on field text
> at
> org.apache.solr.search.facet.UnInvertedField.<init>(UnInvertedField.java:194)
> at
> org.apache.solr.search.facet.UnInvertedField.getUnInvertedField(UnInvertedField.java:595)
> at
> org.apache.solr.search.SolrIndexSearcher$1.regenerateItem(SolrIndexSearcher.java:523)
> at org.apache.solr.search.FastLRUCache.warm(FastLRUCache.java:163)
> at
> org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:2320)
> at org.apache.solr.core.SolrCore$4.call(SolrCore.java:1851)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: Too many values for
> UnInvertedField faceting on field text
> at org.apache.lucene.uninverting.DocTermOrds.uninvert(DocTermOrds.java:489)
> at
> org.apache.solr.search.facet.UnInvertedField.<init>(UnInvertedField.java:192)
> ... 10 more
>
>