You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2009/01/08 20:22:59 UTC
[jira] Commented: (LUCENE-1479) TrecDocMaker skips over documents
when "Date" is missing from documents
[ https://issues.apache.org/jira/browse/LUCENE-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662073#action_12662073 ]
Michael McCandless commented on LUCENE-1479:
--------------------------------------------
Shai, it seems like a doc that has no "Date: XXX" would leave dateStr as null and would then cause an NPE when parseDate is later called? Or am I missing something?
Also I'm getting a compilation error:
{code}
[javac] Compiling 1 source file to /tango/mike/src/lucene.trecdocmaker/build/contrib/benchmark/classes/java
[javac] /tango/mike/src/lucene.trecdocmaker/contrib/benchmark/src/java/org/apache/lucene/benchmark/byTask/feeds/TrecDocMaker.java:190: variable name might not have been initialized
[javac] String name = sb.substring(DOCNO.length(), name.indexOf(TERM_DOCNO, DOCNO.length()));
[javac] ^
[javac] 1 error
{code}
> TrecDocMaker skips over documents when "Date" is missing from documents
> -----------------------------------------------------------------------
>
> Key: LUCENE-1479
> URL: https://issues.apache.org/jira/browse/LUCENE-1479
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Shai Erera
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1479.patch
>
>
> TrecDocMaker skips over Trec documents if they do not have a "Date" line. When such a document is encountered, the code may skip over several documents until the next tag that is searched for is found.
> The result is, instead of reading ~25M documents from the GOV2 collection, the code reads only ~23M (don't remember the actual numbers).
> The fix adds a terminatingTag to read() such that the code looks for prefix, but only until terminatingTag is found. Appropriate changes were made in getNextDocData().
> Patch to follow
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org