You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mike Drob (JIRA)" <ji...@apache.org> on 2017/01/20 21:18:27 UTC
[jira] [Commented] (SOLR-10014) Log a warning when the number of
fields in a core exceeds a configurable value
[ https://issues.apache.org/jira/browse/SOLR-10014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832411#comment-15832411 ]
Mike Drob commented on SOLR-10014:
----------------------------------
I think it makes sense to log the warning on reload. If you are using dynamic fields, as in your example, then we can expect the number of fields to change, right? So initially the warning would not print, and then we would never see it the second time if it's suppressed.
> Log a warning when the number of fields in a core exceeds a configurable value
> ------------------------------------------------------------------------------
>
> Key: SOLR-10014
> URL: https://issues.apache.org/jira/browse/SOLR-10014
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 4.10.4
> Reporter: Shawn Heisey
> Priority: Minor
> Attachments: SOLR-10014.patch
>
>
> When the number of fields in an index gets extremely large, major performance problems can occur. If the number of fields in a core exceeds a configurable number, with a default somewhere around 10000, a warning should be logged when the SolrCore is first created. A decision needs to be made about whether to repeat the warning on core reload ... my instinct is that it should NOT be repeated, but I can see where a repeat might have some value. Logging on reloads as well as startup would likely be easier.
> This was discovered by a Solr user who had a 420MB index with 650K documents, but their applications were abusing dynamic fields to the point where they had about 2 million unique fields in the index. The small size of the index *should* have resulted in extremely fast commit times, but commits were taking about 10 seconds because of what Lucene had to do to handle all those fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org