You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Simon Willnauer (JIRA)" <ji...@apache.org> on 2010/01/07 21:12:54 UTC
[jira] Created: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
-------------------------------------------------------------------------------------------------------
Key: LUCENE-2197
URL: https://issues.apache.org/jira/browse/LUCENE-2197
Project: Lucene - Java
Issue Type: Bug
Components: Analysis
Affects Versions: 3.1
Reporter: Simon Willnauer
Priority: Critical
Fix For: 3.1
With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2197:
------------------------------------
Attachment: LUCENE-2197.patch
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798517#action_12798517 ]
Simon Willnauer commented on LUCENE-2197:
-----------------------------------------
Yonik, would you commit this issue please. I think we agreed on your solution.
simon
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798040#action_12798040 ]
Yonik Seeley commented on LUCENE-2197:
--------------------------------------
Sorry Simon... I think I just got fed up with stuff like this in the JDK over the years (that forces people to write their own implementations for best performance), and you happened to be the closest person at the time :-)
Related: I'm the one who added this to BooleanQuery some time ago:
{code}
/** Returns the list of clauses in this query. */
public List<BooleanClause> clauses() { return clauses; }
{code}
Yes, it probably should also say something like "Don't modify - it may change the query" to the comments.
To the software pedant, that's not safe and would probably be called bad design - but I strongly believe that our API should be for adults, and one should be able to introspect objects like Queries w/o suffering object allocations. We should also continue to develop Lucene for *ourselves*, not for some mythic stupid user... I've seen too many bad design decisions based on "this will confuse users" arguments rather than "this is confusing".
Sometimes it comes down to people trying to solve a class of problems that others aren't even having issues with - I don't ever recall someone accidentally modifying the set after they passed it to the StopFilter, or someone accidentally modifying clauses() from BooleanQuery.
I also disagree with checking all input parameters in many cases (things that could possibly be in someones inner loop and will throw an exception anyway).
Say we have this piece of code:
{code}
boolean checkLength(String str) {
return str.length() < MY_MAX_LENGTH;
}
{code}
I think it's silly to add an explicit null check like so (but you see plenty of code like that):
{code}
boolean checkLength(String str) {
if (str == null) {
throw new RuntimeException("Can't pass checkLength a null string");
}
return str.length() < MY_MAX_LENGTH;
}
{code}
There. Is that reply long enough for you ;-)
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley resolved LUCENE-2197.
----------------------------------
Resolution: Fixed
committed.
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797784#action_12797784 ]
Yonik Seeley commented on LUCENE-2197:
--------------------------------------
Comment from LUCENE-2094 the other issue:
bq. To prevent provided set from being modified while they are in use in a filter the given set is copied and wrapped in an immutable instance of chararrayset.
We should really avoid this type of nannyism in Lucene.
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated LUCENE-2197:
---------------------------------
Attachment: LUCENE-2197.patch
Here's a patch that reverts to the previous behavior of using the set provided.
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797779#action_12797779 ]
Yonik Seeley commented on LUCENE-2197:
--------------------------------------
bq. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
We shouldn't copy it at all.
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798189#action_12798189 ]
Simon Willnauer commented on LUCENE-2197:
-----------------------------------------
bq. Sorry Simon... I think I just got fed up with stuff like this in the JDK over the years (that forces people to write their own implementations for best performance), and you happened to be the closest person at the time
:) no worries, thanks for the reply!
bq. To the software pedant, that's not safe and would probably be called bad design - ...
I understand and I can totally see your point. I was kind of separated due to the kind of short "rants" (don't get me wrong). I agree with you that we should not do that in a filter as this constructor could be called very very frequently especially if an analyzer does not implement reusableTokenStream. I would still argue that for an analyzer this is a different story and I would want to keep the code in analyzers copying the set. Classes, instantiated so frequently as filters should not introduce possible bottlenecks while analyzers are usually shared that won't be much of a hassle - any performance police issues with this? :)
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2197) StopFilter should not create a new
CharArraySet if the given set is already an instance of CharArraySet
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797952#action_12797952 ]
Simon Willnauer commented on LUCENE-2197:
-----------------------------------------
bq. Here's a patch that reverts to the previous behavior of using the set provided.
Doesn't seem to lead anywhere to discuss with the performance police when I look at the average size of your comments. :)
This was actually meant to be a "pattern" for analyzer subclasses so I won't be the "immutability" police here. Yonik, will you take this issue and commit?!
bq. We should really avoid this type of nannyism in Lucene.
oh well this seems to me like a void * is / isn't evil discussion - nevermind.
> StopFilter should not create a new CharArraySet if the given set is already an instance of CharArraySet
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2197
> URL: https://issues.apache.org/jira/browse/LUCENE-2197
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 3.1
> Reporter: Simon Willnauer
> Priority: Critical
> Fix For: 3.1
>
> Attachments: LUCENE-2197.patch, LUCENE-2197.patch
>
>
> With LUCENE-2094 a new CharArraySet is created no matter what type of set is passed to StopFilter. This does not behave as documented and could introduce serious performance problems. Yet, according to the javadoc, the instance of CharArraySet should be passed to CharArraySet.copy (which is very fast for CharArraySet instances) instead of "copied" via "new CharArraySet()"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org