You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2009/07/28 23:16:14 UTC
[jira] Created: (LUCENE-1764) Remote Searching w/sort using
SortComparator has explosive FieldCache usage
Remote Searching w/sort using SortComparator has explosive FieldCache usage
---------------------------------------------------------------------------
Key: LUCENE-1764
URL: https://issues.apache.org/jira/browse/LUCENE-1764
Project: Lucene - Java
Issue Type: Bug
Reporter: Hoss Man
Fix For: 2.9
As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
demonstrating this bug currently requires the patches in LUCENE-1749.
See markmiller's comment here...
https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1764) SampleComparable doesn't work well
in contrib/remote tests
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man resolved LUCENE-1764.
------------------------------
Resolution: Fixed
Committed revision 798726.
hashcode/equals changes along what miller described.
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Minor
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1764) Remote Searching w/sort using
SortComparator has explosive FieldCache usage
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736310#action_12736310 ]
Uwe Schindler commented on LUCENE-1764:
---------------------------------------
I think this bug was there before. SortField has the problem that not only SortComparator, also custom parsers that are not singletons (serializable singletons using readResolve()!!!) or do not have equals/hashcode will create new cache entries each time, e.g. this one:
"new SortField("field", new FieldCache.LongParser() { ... });" creates a different parser instance each time and so a new cache entry is created (even in non-remote usage). This is noted in the docs.
> Remote Searching w/sort using SortComparator has explosive FieldCache usage
> ---------------------------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1764) SampleComparable doesn't work well in
contrib/remote tests
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1764:
--------------------------------
Attachment: LUCENE-1764.patch
Not very happy with it, but here is sort of a stub for the type of doc we need.
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Assignee: Mark Miller
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1764.patch
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-1764) SampleComparable doesn't work well
in contrib/remote tests
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reopened LUCENE-1764:
---------------------------------
I think we need to add warnings - so reopening
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Minor
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1764) Remote Searching w/sort using
SortComparator has explosive FieldCache usage
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736299#action_12736299 ]
Hoss Man commented on LUCENE-1764:
----------------------------------
Note: i set this as "fix in 2.9" because it's not clear if this is a bug thta was newly introduced because of all the sorting changes in 2.9 (in which case we should fix it before release), or if it's always had this problem (in which case we can punt on it if no one has a fix)
> Remote Searching w/sort using SortComparator has explosive FieldCache usage
> ---------------------------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: [jira] Commented: (LUCENE-1764) SampleComparable doesn't work
well in contrib/remote tests
Posted by Chris Hostetter <ho...@fucit.org>.
: SortField.equals() and hashCode() contain a hint:
:
: /** Returns true if <code>o</code> is equal to this. If a
: * {@link SortComparatorSource} (deprecated) or {@link
: * FieldCache.Parser} was provided, it must properly
: * implement equals (unless a singleton is always used). */
:
: Maybe we should make this more visible, contain all different SortField
: comparator/parsers and place it in the the setter methods for parser and
: comparators.
SortField doesn't seem like the right place at all -- people constructing
instances of SortField, or calling setter methods of SortField shouldn't
have to care about this at all -- it's people who extend
SortComparatorSource or FieldCache.Parser who need to be aware of these
issues, so shouldn't the class level javadocs for those packages spell it
out?
(ideally those abstract classes would declare hasCode and equals as
abstract to *force* people to implement them ... but ship has sailed)
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: [jira] Commented: (LUCENE-1764) SampleComparable doesn't work well in contrib/remote tests
Posted by Uwe Schindler <uw...@thetaphi.de>.
Sorry,
it is not very big in the docs (and in my opinion at the wrong place), but
SortField.equals() and hashCode() contain a hint:
/** Returns true if <code>o</code> is equal to this. If a
* {@link SortComparatorSource} (deprecated) or {@link
* FieldCache.Parser} was provided, it must properly
* implement equals (unless a singleton is always used). */
Maybe we should make this more visible, contain all different SortField
comparator/parsers and place it in the the setter methods for parser and
comparators.
Also the note should contain that (if singleton), it must be serializable
(implements the protected readResolve()). Example for serializable
singletons are the default parsers in FieldCache.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> Sent: Wednesday, July 29, 2009 1:55 AM
> To: java-dev@lucene.apache.org
> Subject: Re: [jira] Commented: (LUCENE-1764) SampleComparable doesn't work
> well in contrib/remote tests
>
>
> : We prob want a javadoc warning of some kind too though right? Its not
> : immediately obvious that when you switch to using remote, you better
> : have implemented some form of equals/hashcode or you will have a memory
> : leak.
>
> Hmmm, now i'm confused.
> Uwe's comment in the issue said "This is noted in the docs." and i
> beleived him and figured the problem was exclusive to the SampleComparable
> in the test ... but now that i'm *looking* at the docs, i don't see any
> red flags (in SortField, RemoteSearchable, SoreComparator, etc...)
>
> Uwe?
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-1764) SampleComparable doesn't work
well in contrib/remote tests
Posted by Chris Hostetter <ho...@fucit.org>.
: We prob want a javadoc warning of some kind too though right? Its not
: immediately obvious that when you switch to using remote, you better
: have implemented some form of equals/hashcode or you will have a memory
: leak.
Hmmm, now i'm confused.
Uwe's comment in the issue said "This is noted in the docs." and i
beleived him and figured the problem was exclusive to the SampleComparable
in the test ... but now that i'm *looking* at the docs, i don't see any
red flags (in SortField, RemoteSearchable, SoreComparator, etc...)
Uwe?
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1764) SampleComparable doesn't work well
in contrib/remote tests
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736363#action_12736363 ]
Mark Miller commented on LUCENE-1764:
-------------------------------------
We prob want a javadoc warning of some kind too though right? Its not immediately obvious that when you switch to using remote, you better have implemented some form of equals/hashcode or you will have a memory leak.
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Minor
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1764) SampleComparable doesn't work well
in contrib/remote tests
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller resolved LUCENE-1764.
---------------------------------
Resolution: Fixed
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Assignee: Mark Miller
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1764.patch
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1764) Remote Searching w/sort using
SortComparator has explosive FieldCache usage
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736307#action_12736307 ]
Mark Miller commented on LUCENE-1764:
-------------------------------------
Bug was a bit strong I guess - resource hog trap? - you can sidestep this if you implement hashcode/equals on your SortComparator. We need a warning to do that I think.
For the unit test that fails, I've changed the equals/hashcode for the SortComparator to use:
//@Override
public int hashCode() {
return getClass().getName().hashCode();
}
//@Override
public boolean equals(Object other) {
return this.hashCode() == other.hashCode();
}
We have always had the issue, because the comparator has been part of the key, and when you pass a SortField over the wire, you get a new impl of the comparator factory field thats in it.
> Remote Searching w/sort using SortComparator has explosive FieldCache usage
> ---------------------------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1764) SampleComparable doesn't work well in
contrib/remote tests
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated LUCENE-1764:
-----------------------------
Priority: Minor (was: Major)
Summary: SampleComparable doesn't work well in contrib/remote tests (was: Remote Searching w/sort using SortComparator has explosive FieldCache usage)
updated title/severity since there is no generic problem with SortComparators ... the issue is specifically with SampleCamparable.
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Minor
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1764) SampleComparable doesn't work well
in contrib/remote tests
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reassigned LUCENE-1764:
-----------------------------------
Assignee: Mark Miller
> SampleComparable doesn't work well in contrib/remote tests
> ----------------------------------------------------------
>
> Key: LUCENE-1764
> URL: https://issues.apache.org/jira/browse/LUCENE-1764
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Hoss Man
> Assignee: Mark Miller
> Priority: Minor
> Fix For: 2.9
>
>
> As discovered in LUCENE-1749, when using identical instances of a SortComparator you get multiple entries in the FieldCache.
> demonstrating this bug currently requires the patches in LUCENE-1749.
> See markmiller's comment here...
> https://issues.apache.org/jira/browse/LUCENE-1749?focusedCommentId=12735190#action_12735190
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org