You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "dnhatn (via GitHub)" <gi...@apache.org> on 2024/05/06 22:49:39 UTC
[PR] Harden BaseDocValuesFormatTestCase [lucene]
dnhatn opened a new pull request, #13346:
URL: https://github.com/apache/lucene/pull/13346
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Harden BaseDocValuesFormatTestCase [lucene]
Posted by "dnhatn (via GitHub)" <gi...@apache.org>.
dnhatn commented on PR #13346:
URL: https://github.com/apache/lucene/pull/13346#issuecomment-2098808242
Thanks @jpountz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Harden BaseDocValuesFormatTestCase [lucene]
Posted by "dnhatn (via GitHub)" <gi...@apache.org>.
dnhatn commented on code in PR #13346:
URL: https://github.com/apache/lucene/pull/13346#discussion_r1592732847
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseDocValuesFormatTestCase.java:
##########
@@ -1414,6 +1414,45 @@ protected void assertDVIterate(Directory dir) throws IOException {
ir.close();
}
+ protected void compareStoredFieldWithSortedNumericsDV(
+ DirectoryReader directoryReader, String storedField, String dvField) throws IOException {
+ for (LeafReaderContext leaf : directoryReader.leaves()) {
+ LeafReader reader = leaf.reader();
+ StoredFields storedFields = reader.storedFields();
+ SortedNumericDocValues docValues = reader.getSortedNumericDocValues(dvField);
+ if (docValues == null) {
+ // no stored values at all
+ for (int doc = 0; doc < reader.maxDoc(); doc++) {
+ assertArrayEquals(new String[0], storedFields.document(doc).getValues(storedField));
+ }
+ continue;
+ }
+ for (int doc = 0; doc < reader.maxDoc(); doc++) {
+ String[] storedValues = storedFields.document(doc).getValues(storedField);
+ if (storedValues.length == 0) {
+ assertFalse(docValues.advanceExact(doc));
+ continue;
+ }
+ switch (random().nextInt(3)) {
+ case 0 -> assertEquals(doc, docValues.nextDoc());
+ case 1 -> assertEquals(doc, docValues.advance(doc));
+ default -> assertTrue(docValues.advanceExact(doc));
+ }
+ assertEquals(doc, docValues.docID());
+ assertEquals(storedValues.length, docValues.docValueCount());
+ int repeats = 1 + random().nextInt(3);
+ for (int r = 0; r < repeats; r++) {
+ if (r > 0 || random().nextBoolean()) {
+ assertTrue(docValues.advanceExact(doc));
+ }
+ for (int v = 0; v < docValues.docValueCount(); v++) {
+ assertEquals(storedValues[v], Long.toString(docValues.nextValue()));
+ }
+ }
+ }
+ }
Review Comment:
++, I pushed https://github.com/apache/lucene/pull/13346/commits/59686c4692b5a6ee63f9f358b68ac923755d44b4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Harden BaseDocValuesFormatTestCase [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13346:
URL: https://github.com/apache/lucene/pull/13346#discussion_r1592546383
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseDocValuesFormatTestCase.java:
##########
@@ -1414,6 +1414,45 @@ protected void assertDVIterate(Directory dir) throws IOException {
ir.close();
}
+ protected void compareStoredFieldWithSortedNumericsDV(
+ DirectoryReader directoryReader, String storedField, String dvField) throws IOException {
+ for (LeafReaderContext leaf : directoryReader.leaves()) {
+ LeafReader reader = leaf.reader();
+ StoredFields storedFields = reader.storedFields();
+ SortedNumericDocValues docValues = reader.getSortedNumericDocValues(dvField);
+ if (docValues == null) {
+ // no stored values at all
+ for (int doc = 0; doc < reader.maxDoc(); doc++) {
+ assertArrayEquals(new String[0], storedFields.document(doc).getValues(storedField));
+ }
+ continue;
+ }
+ for (int doc = 0; doc < reader.maxDoc(); doc++) {
+ String[] storedValues = storedFields.document(doc).getValues(storedField);
+ if (storedValues.length == 0) {
+ assertFalse(docValues.advanceExact(doc));
+ continue;
+ }
+ switch (random().nextInt(3)) {
+ case 0 -> assertEquals(doc, docValues.nextDoc());
+ case 1 -> assertEquals(doc, docValues.advance(doc));
+ default -> assertTrue(docValues.advanceExact(doc));
+ }
+ assertEquals(doc, docValues.docID());
+ assertEquals(storedValues.length, docValues.docValueCount());
+ int repeats = 1 + random().nextInt(3);
+ for (int r = 0; r < repeats; r++) {
+ if (r > 0 || random().nextBoolean()) {
+ assertTrue(docValues.advanceExact(doc));
+ }
+ for (int v = 0; v < docValues.docValueCount(); v++) {
+ assertEquals(storedValues[v], Long.toString(docValues.nextValue()));
+ }
+ }
+ }
+ }
Review Comment:
This loop looks good to me, I'm contemplating doing a second loop using advanceExact every Nth document in case there are bugs lurking in case we ignore some documents?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Harden BaseDocValuesFormatTestCase [lucene]
Posted by "dnhatn (via GitHub)" <gi...@apache.org>.
dnhatn merged PR #13346:
URL: https://github.com/apache/lucene/pull/13346
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org