You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by "Digy (JIRA)" <ji...@apache.org> on 2008/08/12 21:39:44 UTC
[jira] Created: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
NUnit test for Index/TestStressIndexing2
----------------------------------------
Key: LUCENENET-143
URL: https://issues.apache.org/jira/browse/LUCENENET-143
Project: Lucene.Net
Issue Type: Bug
Environment: Lucene.Net 2.3.1
Reporter: Digy
Priority: Minor
Attachments: TestStressIndexing2.patch
Following patch resolves a bug, but this test continues to fail.
DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Digy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Digy closed LUCENENET-143.
--------------------------
Resolution: Fixed
Assignee: Digy
TestStressIndexing2.patch is applied.
DIGY
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Assignee: Digy
> Priority: Minor
> Attachments: FieldsReader.patch, TestStressIndexing2.patch, TestStressIndexing2.patch, TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Digy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Digy updated LUCENENET-143:
---------------------------
Attachment: TestStressIndexing2.patch
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Doug Sale (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Sale updated LUCENENET-143:
--------------------------------
Attachment: TestStressIndexing2.patch
This patch incorporates the runtime fix supplied by Digy as well as fixing the console "debug" in the tests and updating the supplied comparator to check both field name and field value.
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: TestStressIndexing2.patch, TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Doug Sale (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622271#action_12622271 ]
Doug Sale commented on LUCENENET-143:
-------------------------------------
Please disregard the last line in my previous comment. I cannot replicate that condition.
However, I have modified the "debug" code in the Lucene.Net version of the tests to correctly print out the Fields in each Document when they are found to be unequal. I have also modified the code in the Java version of the test to always print out the Fields.
Somehow (in the Java version of the test) even when there are duplicate Fields with the same name, said Fields are always in the same order for both Lists. This is odd, considering that the explicitly defined comparator only looks at Field.name(). I've checked the other data structures in which the Fields are stored (ArrayList inside of Document), but have found no other implicit or explicit ordering that would explain this.
So, I modified the comparator to check both field name and field contents to ensure that each Field List for the 2 Documents are in order prior to comparison. Both tests (TestRandom() and TestMultiConfig()) both clear VerifyEquals(Document d1, Document d2) and now are failing in VerifyEquals(TermFreqVector[] d1, TermFreqVector[] d2).
I will provide a patch that includes these changes (fixing the "debug" code and updating the comparator) as well as incorporating Digy's patch, shortly.
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Doug Sale (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622252#action_12622252 ]
Doug Sale commented on LUCENENET-143:
-------------------------------------
This test is incorrect. I've also been investigating it.
TestRandom()
- uses multiple threads to randomly generate an index
- collects all the documents in a dictionary
- copies the documents into a 2nd index
- compares the indexes for equality, doc-by-doc, field-by-field
After the run-time error fix (swapping the position of the id field in an ArrayList) - the test fails when comparing the fields in the documents.
The fields are placed in lists and sorted prior to comparison. However, the sort only compares field names. Some documents, due to the random nature of their generation, have multiple fields with the same name. So, you could have 2 equivalent documents represented as:
Doc 1: f2=A A C ; f3=D E I H E I B J D E ; f3=J A I F G E I I D J ; id=0;
Doc 2: f2=A A C ; f3=J A I F G E I I D J ; f3=D E I H E I B J D E ; id=0;
When the fields are compared in this order, the equivalency test fails. <f3=D E I H E I B J D E> != <f3=J A I F G E I I D J>
Checking out the Lucene version of the test, when the documents' fields are compared, there is only 1 field in each List (the "id" field). I'm looking into the discrepancy now.
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Doug Sale (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Sale updated LUCENENET-143:
--------------------------------
Attachment: TestStressIndexing2.patch
Resolves outstanding issues. Incorporates fixes from earlier "TestStressIndexing2.patch" patches.
These tests create docs composed of random fields and add them to an index. The index documents are then copied, field-by-field to another index. The indexes are then compared for equality. (This is all done with multiple threads and myriad different knobs twisted this way and that...)
In the Java code, there were "random" orderings of the fields, which were sometimes sorted at different points depending on chance.
However, the randomizations in the Java code are providing the completely consistent results, as the logic of the original tests could not work otherwise.
The fixes incorporated in this patch include (introduced prior):
- a fix to the random ordering of the fields (not ported properly)
- a fix to the IComparator to ensure that fields are ordered by name and value (otherwise, fields with the same values were in an undefined order)
The recent fix
- removing the field sort when copying from original doc to new doc (when multiple fields of the same name exist in a doc, changing their order of addition changes the term position and offsets, causing the tests to fail)
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: FieldsReader.patch, TestStressIndexing2.patch, TestStressIndexing2.patch, TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Digy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Digy updated LUCENENET-143:
---------------------------
Attachment: FieldsReader.patch
There is a bug in "FieldsReader.Doc".
After applying this patch and TestStressIndexing2.patch(by Doug) and ensuring that "sameFieldOrder" in TestStressIndexing2 is always true, this test passes.
PS: As Doug pointed out, I don't understand how this test passes in Java without ordering the fields in documents
DIGY
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: FieldsReader.patch, TestStressIndexing2.patch, TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (LUCENENET-143) NUnit test for
Index/TestStressIndexing2
Posted by "Digy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENENET-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647958#action_12647958 ]
Digy commented on LUCENENET-143:
--------------------------------
FieldsReader.patch applied.
DIGY
> NUnit test for Index/TestStressIndexing2
> ----------------------------------------
>
> Key: LUCENENET-143
> URL: https://issues.apache.org/jira/browse/LUCENENET-143
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Priority: Minor
> Attachments: FieldsReader.patch, TestStressIndexing2.patch, TestStressIndexing2.patch, TestStressIndexing2.patch
>
>
> Following patch resolves a bug, but this test continues to fail.
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.