You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Grant Ingersoll (JIRA)" <ji...@apache.org> on 2009/03/30 18:44:51 UTC

[jira] Commented: (SOLR-1095) StopFilterFactory should not use HashSet

    [ https://issues.apache.org/jira/browse/SOLR-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693832#action_12693832 ] 

Grant Ingersoll commented on SOLR-1095:
---------------------------------------

I should also note this is probably more noticeable in this particular case due to the fact that the data in question has a lot of small fields.

> StopFilterFactory should not use HashSet
> ----------------------------------------
>
>                 Key: SOLR-1095
>                 URL: https://issues.apache.org/jira/browse/SOLR-1095
>             Project: Solr
>          Issue Type: Bug
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Critical
>
> Rev 680935 of StopFilterFactory introduced some changes that can cause poor performance do to the use of HashSet instead of CharArraySet in the inform() method.  I noticed this while profiling http://www.lucidimagination.com/search/document/f46f0edfc439356c/solr_user_upgrade_from_1_2_to_1_3_gives_3x_slowdown whereby 20% of the time spent in indexing occurred in Collections.addAll() in the Lucene StopFilter due to the execution of the else clause in the lines:
> {code}
> if (stopWords instanceof CharArraySet) {
>       this.stopWords = (CharArraySet)stopWords;
>     } else {
>       this.stopWords = new CharArraySet(stopWords.size(), ignoreCase);
>       this.stopWords.addAll(stopWords);
>     }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.