You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2021/06/14 08:19:03 UTC
[GitHub] [lucene] jpountz commented on a change in pull request #180: LUCENE-9959: [WIP] Add non thread local based API for term vector reader usage
jpountz commented on a change in pull request #180:
URL: https://github.com/apache/lucene/pull/180#discussion_r650714447
##########
File path: lucene/core/src/java/org/apache/lucene/index/BaseCompositeReader.java
##########
@@ -118,6 +121,65 @@ public final Fields getTermVectors(int docID) throws IOException {
return subReaders[i].getTermVectors(docID - starts[i]); // dispatch to subreader
}
+ private class CompositeTermVectorsReader extends TermVectorsReader {
+ List<TermVectorsReader> termVectorsReaders;
+
+ public CompositeTermVectorsReader(List<TermVectorsReader> termVectorsReaders) {
+ this.termVectorsReaders = termVectorsReaders;
+ }
+
+ @Override
+ public Fields get(int doc) throws IOException {
+ ensureOpen();
+ final int i = readerIndex(doc); // find subreader num
+ return termVectorsReaders.get(i).get(doc - starts[i]); // dispatch to subreader
+ }
+
+ @Override
+ public void checkIntegrity() throws IOException {
+ termVectorsReaders.stream()
+ .forEach(
+ r -> {
+ try {
+ r.checkIntegrity();
+ } catch (IOException e) {
+ throw new UncheckedIOException(e);
+ }
+ });
+ }
+
+ @Override
+ public TermVectorsReader clone() {
+ List<TermVectorsReader> newTermVectorReaders =
+ termVectorsReaders.stream().map(r -> r.clone()).collect(Collectors.toList());
Review comment:
nit: I like method refs better when applicable
```suggestion
termVectorsReaders.stream().map(TermVectorsReader::clone).collect(Collectors.toList());
```
##########
File path: lucene/core/src/java/org/apache/lucene/index/BaseCompositeReader.java
##########
@@ -118,6 +121,65 @@ public final Fields getTermVectors(int docID) throws IOException {
return subReaders[i].getTermVectors(docID - starts[i]); // dispatch to subreader
}
+ private class CompositeTermVectorsReader extends TermVectorsReader {
+ List<TermVectorsReader> termVectorsReaders;
+
+ public CompositeTermVectorsReader(List<TermVectorsReader> termVectorsReaders) {
+ this.termVectorsReaders = termVectorsReaders;
+ }
+
+ @Override
+ public Fields get(int doc) throws IOException {
+ ensureOpen();
+ final int i = readerIndex(doc); // find subreader num
+ return termVectorsReaders.get(i).get(doc - starts[i]); // dispatch to subreader
+ }
+
+ @Override
+ public void checkIntegrity() throws IOException {
+ termVectorsReaders.stream()
+ .forEach(
+ r -> {
+ try {
+ r.checkIntegrity();
+ } catch (IOException e) {
+ throw new UncheckedIOException(e);
+ }
+ });
Review comment:
nit: a `for` loop would be simpler than streams here? :)
##########
File path: lucene/core/src/java/org/apache/lucene/index/IndexReader.java
##########
@@ -310,6 +311,9 @@ public final int hashCode() {
*/
public abstract Fields getTermVectors(int docID) throws IOException;
+ /** Get TermVectorsReader from this index. */
+ public abstract TermVectorsReader getTermVectorsReaderNonThreadLocal();
Review comment:
We shouldn't expose `TermVectorsReader` directly but instead create a new class that has a `public Fields get(int doc) throws IOException;` method like `TermVectorsReader` but none of the `clone`/`getMergeInstance`/`checkIntegrity` logic, which only belongs to codec APIs.
One way to do this would be to create a class e.g. called `TermVectors` that only has `public Fields get(int doc) throws IOException;`, use this class in `IndexReader`, make `TermVectorsReader` extend `TermVectors`, and then upgrade the return type of `getTermVectorsReaderNonThreadLocal` to `TermVectorsReader` in `CodecReader`.
##########
File path: lucene/core/src/java/org/apache/lucene/index/BaseCompositeReader.java
##########
@@ -118,6 +121,65 @@ public final Fields getTermVectors(int docID) throws IOException {
return subReaders[i].getTermVectors(docID - starts[i]); // dispatch to subreader
}
+ private class CompositeTermVectorsReader extends TermVectorsReader {
+ List<TermVectorsReader> termVectorsReaders;
+
+ public CompositeTermVectorsReader(List<TermVectorsReader> termVectorsReaders) {
+ this.termVectorsReaders = termVectorsReaders;
+ }
+
+ @Override
+ public Fields get(int doc) throws IOException {
+ ensureOpen();
+ final int i = readerIndex(doc); // find subreader num
+ return termVectorsReaders.get(i).get(doc - starts[i]); // dispatch to subreader
+ }
+
+ @Override
+ public void checkIntegrity() throws IOException {
+ termVectorsReaders.stream()
+ .forEach(
+ r -> {
+ try {
+ r.checkIntegrity();
+ } catch (IOException e) {
+ throw new UncheckedIOException(e);
+ }
+ });
+ }
+
+ @Override
+ public TermVectorsReader clone() {
+ List<TermVectorsReader> newTermVectorReaders =
+ termVectorsReaders.stream().map(r -> r.clone()).collect(Collectors.toList());
+
+ return new CompositeTermVectorsReader(newTermVectorReaders);
+ }
+
+ @Override
+ public void close() throws IOException {
+ termVectorsReaders.stream()
+ .forEach(
+ r -> {
+ try {
+ r.close();
+ } catch (IOException e) {
+ throw new UncheckedIOException(e);
+ }
+ });
Review comment:
we should use `IOUtils#close` for this, which will make sure to keep closing other resources if one of them throws an IOException upon close
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org