You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Ard Schrijvers (JIRA)" <ji...@apache.org> on 2007/08/06 14:13:59 UTC
[jira] Created: (JCR-1051) MatchAllScorer calculateDocFilter() bug
MatchAllScorer calculateDocFilter() bug
---------------------------------------
Key: JCR-1051
URL: https://issues.apache.org/jira/browse/JCR-1051
Project: Jackrabbit
Issue Type: Bug
Components: indexing
Affects Versions: 1.3
Reporter: Ard Schrijvers
Fix For: 1.4
In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
Change
bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
to
bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
-----------------------------------------------------------------------
TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
try {
TermDocs docs = reader.termDocs();
try {
while (terms.term() != null
&& terms.term().field() == FieldNames.PROPERTIES
&& terms.term().text().startsWith(field)) {
docs.seek(terms);
while (docs.next()) {
docFilter.set(docs.doc());
}
terms.next();
}
} finally {
docs.close();
}
} finally {
terms.close();
}
-----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: Reopened: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by Christoph Kiehl <ch...@sulu3000.de>.
Marcel Reutegger (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Marcel Reutegger reopened JCR-1051:
> -----------------------------------
>
> Assignee: Marcel Reutegger (was: Christoph Kiehl)
>
> We should avoid creating the named value in the while loop.
So true ;) Glad you're are back!
Cheers,
Christoph
[jira] Assigned: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Christoph Kiehl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christoph Kiehl reassigned JCR-1051:
------------------------------------
Assignee: Christoph Kiehl
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Assignee: Christoph Kiehl
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Marcel Reutegger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcel Reutegger resolved JCR-1051.
-----------------------------------
Resolution: Fixed
Fixed in revision: 565316
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Assignee: Marcel Reutegger
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Christoph Kiehl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christoph Kiehl updated JCR-1051:
---------------------------------
Attachment: MatchAllScorerWithTest.patch
Patch looks good. If no one objects I will commit the patch together with the proposed addition to the testcase.
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Ard Schrijvers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ard Schrijvers updated JCR-1051:
--------------------------------
Attachment: MatchAllScorer.java.patch
fix
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Marcel Reutegger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcel Reutegger reopened JCR-1051:
-----------------------------------
Assignee: Marcel Reutegger (was: Christoph Kiehl)
We should avoid creating the named value in the while loop.
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Assignee: Marcel Reutegger
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Christoph Kiehl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christoph Kiehl resolved JCR-1051.
----------------------------------
Resolution: Fixed
Patch applied in revision 563673. Thanks Ard!
Please post a mail to the developers list to discuss optimization of MatchAllScorer. I'm very interested in your suggestions.
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.3
> Reporter: Ard Schrijvers
> Assignee: Christoph Kiehl
> Fix For: 1.4
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-1051) MatchAllScorer calculateDocFilter() bug
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated JCR-1051:
-------------------------------
Affects Version/s: 1.2.1
1.2.2
1.2.3
1.3.1
Fix Version/s: (was: 1.4)
1.3.2
Merged to the 1.3 branch in revision 577816.
> MatchAllScorer calculateDocFilter() bug
> ---------------------------------------
>
> Key: JCR-1051
> URL: https://issues.apache.org/jira/browse/JCR-1051
> Project: Jackrabbit
> Issue Type: Bug
> Components: indexing
> Affects Versions: 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1
> Reporter: Ard Schrijvers
> Assignee: Marcel Reutegger
> Fix For: 1.3.2
>
> Attachments: MatchAllScorer.java.patch, MatchAllScorerWithTest.patch
>
>
> In MatchAllScorer.calculateDocFilter(), When you have just two nodes, with different properties, like "myprop" and "myprop2", and you have an xpath String xpath = "//*[@myprop], you get both nodes back (to be precise, you'll get every node that has a property that startswith "myprop")
> You can reproduce it by changing the SimpleQueryTest.testIsNotNull() a little:
> Change
> bar.setProperty("text", "the quick brown fox jumps over the lazy dog.");
> to
> bar.setProperty("mytextwhichstartswithmytext", "the quick brown fox jumps over the lazy dog.");
> Now the test with xpath = "//*[@jcr:primaryType='nt:unstructured' and @mytext]"; fails because 2 results. I did test for the trunk and tag 1.3.1 and both have the same problem. I have attached MatchAllScorer.java.patch in this mail, or should I create a JIRA issue for this?
> Furthermore I would like to discuss a different implementation for the MatchAllScorer, because IMHO the current calculateDocFilter() becomes slow pretty fast (see bottom email the code part i am referring to: if you have 100.000 docs with "mytext" property, and you query [@mytext] the loop below is executed at least 100.000 times). I think it might be out of scope for the user-list, or is the user-list the place to discuss something like this?
> -----------------------------------------------------------------------
> TermEnum terms = reader.terms(new Term(FieldNames.PROPERTIES, field));
> try {
> TermDocs docs = reader.termDocs();
> try {
> while (terms.term() != null
> && terms.term().field() == FieldNames.PROPERTIES
> && terms.term().text().startsWith(field)) {
> docs.seek(terms);
> while (docs.next()) {
> docFilter.set(docs.doc());
> }
> terms.next();
> }
> } finally {
> docs.close();
> }
> } finally {
> terms.close();
> }
> -----------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.