[jira] [Created] (LUCENE-6868) ParallelLeafReader.getTermVectors can indirectly load TVs multiple times

["David Smiley (JIRA)" <ji...@apache.org>]

David Smiley created LUCENE-6868:
------------------------------------

             Summary: ParallelLeafReader.getTermVectors can indirectly load TVs multiple times
                 Key: LUCENE-6868
                 URL: https://issues.apache.org/jira/browse/LUCENE-6868
             Project: Lucene - Core
          Issue Type: Improvement
          Components: core/index, core/termvectors
            Reporter: David Smiley


ParallelLeafReader has a getTermVectors(docId) implementation that loops over each field it has in a loop and calls getTermVector(docId,fieldName).  But the implementation of that will load all term vectors for all fields in that reader, yet ParallelLeafReader only wants one.  The effect is an O(n^2) where 'n' is the number of fields, when we could get O(n) if we do it right. PLR should call getTermVectors(docId) (not referring to a specific field) for each of it's readers and then aggregate them.

This wouldn't be such a problem if our term vector API/Codec was improved to not load all term vectors for all fields from disk at once.

Found via randomized-testing of IndexWriter auto-picking ParallelAtomicReader along with a test I have that asserts TVs aren't fetched for a doc more than once.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org