You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jimczi (via GitHub)" <gi...@apache.org> on 2026/10/15 12:52:09 UTC

[PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

jimczi opened a new pull request, #16847:
URL: https://github.com/apache/lucene/pull/16847

   **Draft — opening this to show the approach and get early feedback, not as a finished change.**
   
   This is an attempt at letting a single index hold more than `Integer.MAX_VALUE - 128`
   documents. Today a doc is an `int` everywhere, so once the segments together pass ~2.1B
   docs there's nowhere to put the number and you have to shard. I wanted to see how far you
   get by widening only the whole-index doc id to `long` and leaving everything inside a
   segment alone.
   
   The idea the whole branch rests on: a doc id inside a segment stays an `int` in
   `[0, maxDoc)`, and a segment still can't exceed `MAX_DOCS`. Only the *composite* doc id —
   `leaf.docBase + leafDocId` — becomes a `long`. That line falls between the leaf and the
   reader above it, so codecs and every `*Format` never see a `long` and there's no file
   format change; existing indices read back unchanged, and an index under 2^31 docs pays
   nothing.
   
   What that meant in practice:
   
   - `IndexReader` gets `long totalMaxDoc()` / `totalNumDocs()`. The old `int maxDoc()` /
     `numDocs()` still work below 2^31 but now throw `ArithmeticException` once the real count
     doesn't fit, instead of quietly overflowing. Leaf readers keep the `int` versions.
   - `LeafReaderContext.docBase` is a `long`, with one narrowing helper (`toLeafDocId(long)`)
     that asserts the doc really belongs to the leaf, so the `long`→`int` cast only happens in
     one place.
   - `ScoreDoc.doc` and `FieldDoc` carry a `long`, and that flows through `HitQueue`,
     `TopDocs`/`TopFieldDocs`, `searchAfter(...)` and `explain(...)`. This is the part that
     actually breaks callers: anything reading `scoreDoc.doc` into an `int`, or implementing a
     `Collector` / `FieldComparator`, has to recompile and retype. It's the main reason this
     can't be backported.
   - There are two limits now. The per-segment one (`MAX_DOCS`, `int`) is unchanged. A new
     whole-index one (`MAX_TOTAL_DOCS`, currently `2^37` ≈ 137B) caps the total; I picked a
     bounded number rather than `Long.MAX_VALUE` mostly so merge/accounting stay sane — happy
     to be argued out of the exact value, it's deliberately a round-ish placeholder.
   - The top-scores heap keeps the existing packed `(score, doc)`-in-a-`long` implementation
     for indices up to 2^31 so the common case doesn't regress, and switches to a `long`-doc
     heap only beyond that.
   - The composite "treat the whole reader as one leaf" helpers (`MultiTerms`, `MultiBits`,
     `MultiDocValues`) can't represent a `> 2^31` space in their int APIs, so they now fail
     fast with a "consume the index per-leaf instead" message rather than overflowing.
   
   Testing is awkward because you obviously can't index billions of docs in a unit test, so
   `TestSearchBeyondMaxInt` fakes it: a couple of real one-doc segments are wrapped in
   `FilterLeafReader`s that report an inflated `maxDoc` and placed ahead of the real leaf in a
   `MultiReader`, which pushes that leaf's `docBase` — and its hits' doc ids — past 2^31 with
   no large allocation. From there it checks that hits come back as genuine `long`s, that
   global-doc dispatch (`subIndex`, `storedFields.document(long)`) lands on the right leaf,
   that `searchAfter` paging works across the boundary for score / field / `_doc` sort, and
   that the fail-fast helpers throw the clear message. The rest of the core suite and every
   touched module pass as-is, and I turned `CollectionIncompatibleType` into a build error to
   catch the `Set<Integer>.contains(longDoc)` autobox traps the widening invites.
   
   Rough edges I already know about, and why it's still a draft:
   
   - A few structures are still sized to the whole index — grouping's group-heads keep a dense
     per-leaf `FixedBitSet`, which is `O(totalMaxDoc)` memory. Fine functionally, wants a
     sparse form before anyone runs this near the top of the range.
   - `ReaderUtil.partitionByLeaf` is still `int`-bounded (nothing calls it past 2^31 yet).
   - A handful of peripheral spots (some `suggest`, `spatial-extras`, `monitor`) still throw a
     generic `ArithmeticException` past 2^31 instead of the nice per-leaf message. They fail
     safely; I just haven't routed them through the shared guard.
   
   Mostly I'd like to know whether the leaf-int / global-long split is the right shape before
   polishing any of this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6290546941

   I did another pass on merges and merge policies. A segment still uses an int for its doc ids. So a single merge can never go over `MAX_DOCS`, even if the whole index now holds more. Tiered already capped its merges for that. I made `LogMergePolicy` do the same, so it leaves a few more segments instead of overflowing.
   
   Soft deletes count here too, and I missed at first that the count can be an approximation. The policy estimates how many deletes a merge will reclaim, but a merge can keep more than that. So the guard looks at the docs the merge really writes, not the estimate. When the retention only moves forward this stays on the safe side, since the retained set only shrinks by merge time.
   
   I also added a guard in `IndexWriter`. It fails the merge early with a clear message if a policy still picks one that is too big. And `BaseMergePolicyTestCase` now checks this, so a custom merge policy catches it in its own tests.
   
   @iverase thanks for testing this, it pushed me to go through a few more cases while I was here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #16847:
URL: https://github.com/apache/lucene/pull/16847#discussion_r4305173408


##########
gradle/validation/ast-grep/rules/java-patterns.yml:
##########
@@ -133,3 +133,32 @@ constraints:
 severity: error
 message: Avoid pre-sized arrays in Collection#toArray calls
 note: Prefer `toArray($TYPE[]::new)` or `toArray(new $TYPE[0])`
+---
+# yaml-language-server: $schema=https://raw.githubusercontent.com/ast-grep/ast-grep/refs/heads/main/schemas/java_rule.json
+id: raw-global-doc-id-narrowing
+language: java
+rule:
+  any:
+    - pattern: (int) ($DOC - $CTX.docBase)
+    - pattern: (int) ($DOC - docBase)

Review Comment:
   do you mind adding a trivial test? it helps to maintain them and upgrade ast-grep and so on.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on code in PR #16847:
URL: https://github.com/apache/lucene/pull/16847#discussion_r4322284303


##########
lucene/core/src/java/org/apache/lucene/document/LatLonPoint.java:
##########
@@ -424,7 +424,9 @@ public static TopFieldDocs nearest(
       if (points != null) {
         totalHits += points.getDocCount();
         readers.add(points);
-        docBases.add(leaf.docBase);
+        // NearestNeighbor addresses the composite doc space with int; point-nearest does not
+        // support a reader with more than Integer.MAX_VALUE docs.
+        docBases.add(Math.toIntExact(leaf.docBase));

Review Comment:
   Done. Switched to `LongArrayList` and widened the composite doc id through `NearestNeighbor`, so point nearest-neighbor search works past 2^31 now. Pushed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "github-actions[bot] (via GitHub) via issues" <is...@lucene.apache.org>.
github-actions[bot] commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6995726900

   This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "iverase (via GitHub)" <gi...@apache.org>.
iverase commented on code in PR #16847:
URL: https://github.com/apache/lucene/pull/16847#discussion_r4315406058


##########
lucene/core/src/java/org/apache/lucene/document/LatLonPoint.java:
##########
@@ -424,7 +424,9 @@ public static TopFieldDocs nearest(
       if (points != null) {
         totalHits += points.getDocCount();
         readers.add(points);
-        docBases.add(leaf.docBase);
+        // NearestNeighbor addresses the composite doc space with int; point-nearest does not
+        // support a reader with more than Integer.MAX_VALUE docs.
+        docBases.add(Math.toIntExact(leaf.docBase));

Review Comment:
   I think using a LongArrayList should be possible so we can get over this limitation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6285300329

   This is already handled, just not documented. The merge policies cap every merged segment at `MAX_DOCS`, so `forceMerge` can't build an oversized segment, it just stops short of `maxNumSegments` when the live docs don't fit. I updated the javadoc to say so.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6260185845

   Thanks @rmuir, I really like this idea. I gave it a try and it comes out cleaner than the `total*()` methods, so I reworked the branch onto it.
   
   `IndexReader#maxDoc()/numDocs()` return `long` now. `LeafReader` keeps the `int` under `maxLeafDoc()/numLeafDocs()`. I also put `maxLeafDoc()/numLeafDocs()` on `LeafReaderContext`, so per-segment code can size a bitset straight from the context without calling `reader()` just to get the count.
   
   It breaks the way you described. Anything that needs an `int` stops compiling until you use the leaf accessor, so nothing narrows silently. It's a wide change but almost all of it is mechanical. Everything is green locally and I pushed it to the branch.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on code in PR #16847:
URL: https://github.com/apache/lucene/pull/16847#discussion_r4312744029


##########
lucene/core/src/java/org/apache/lucene/index/IndexReader.java:
##########
@@ -294,22 +294,25 @@ public final int hashCode() {
    *
    * <p><b>NOTE</b>: This operation may run in O(maxDoc). Implementations that can't return this
    * number in constant-time should cache it.
+   *
+   * <p>Access the per-segment {@code int} count with {@link LeafReader#numLeafDocs()}.
    */
-  public abstract int numDocs();
+  public abstract long numDocs();

Review Comment:
   Yeah agreed, it needs the same treatment as maxDoc and numDocs. We can apply the same idea there too. I just put up a commit for it.
   
   `docFreq(Term)` and `getDocCount(String)` return `long` on `IndexReader` now, since across a composite reader they can go past an int. The per-segment counts stay int, you get them from `leafDocFreq()` and `getLeafDocCount()` on `LeafReader`.
   
   The `TermStats` and `FieldStats` records were already long so I missed that the feeders were still int. `TermStates` now accumulates docFreq as a long and `IndexSearcher.termStats` takes a long. There was also a silent overflow in the static `PointValues.getDocCount(reader, field)` helper that sums across leaves, so that one returns long now. The per-segment `getDocCount()` stays int.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "iverase (via GitHub)" <gi...@apache.org>.
iverase commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6270960201

   We might need to change the contract of `IndexWriter#forceMerge(int maxNumSegments)` as it some cases it won't be able to comply. E.g. An index with 4B documents cannot be force merged to a single segment. The current contract says: 
   ```
   Forces merge policy to merge segments until there are {@code <= maxNumSegments}
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6268703763

   > but maybe it would be less disruptive, at least in terms of files/lines changed, than adding the new `leafMaxDoc`?
   
   IMO this would be optimizing for the wrong thing. Giving the user clear compile-time breaks is going to result in more changes on our side, but they should mostly mechanical/easy... aka noise.
   
   I knew this would "explode" the PR with such changes when posting the suggestion. I think easy/clear compile-time breakage is many times better than less-type-safe corner-case runtime-breakage (especially if it only happens with large integers above 2B)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6244052541

   Hi @jimczi, I think it is a good idea and thanks for prototyping. It is gonna be messy either way due to the nature of the change, so it helps to see.
   
   The biggest concern I have, is the user-facing API. I think just adding the new `long` methods to `IndexReader`, whilst keeping the old `int`-based methods is a bit confusing to the end-user and error-prone. Adding numeric safety checks such as `toIntExact` is a good thing but I'm guessing most users would not hit that in their tests.
   
   So I'm wondering, how messy would it be if the `int`-based methods were instead moved to `LeafReader`? The methods on `IndexReader` would just support `long`. Although it is "breaking" to the user-code, I think it would "break" users in the best possible way: they'd have full compile-time safety to fix their code.
   
   ```java
     /**
      * Returns the number of documents in this index.
      *
      * <p><b>NOTE</b>: This operation may run in O(maxDoc). Implementations that can't return this
      * number in constant-time should cache it.
      */
     public abstract long numDocs();
   
     /**
      * Returns one greater than the largest possible document number. 
      */
     public abstract long maxDoc();
   ```
   
   So for most users that are just searching documents, the compiler would guide them to change `int`'s to `long`'s and then their app would work as expected, but with the 2^31 limit removed.
   
   For users that are doing per-segment stuff (e.g. custom components / datastructures), they'd also have a compile-time break and they'd have in LeafReader the `int`-based methods to help ensure safety (maybe think of a new name such as maxLeafDoc() or whatever).
   
   I think this might be one of those cases where a clear break would help the user, if we do it correctly.
   
   Please note: this is just highly theoretical idea as I haven't tried it out or anything to see if the API would be viable in this way.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6266686310

   +1 -- a few of us had a nice discussion about similar ideas at CoC (changing signature of existing methods from int to long). We also talked about whether `CompositeReader` is still needed, and I think the idea was -- no, but removing it is a lot of trouble with no real improvement...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "iverase (via GitHub)" <gi...@apache.org>.
iverase commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6288460626

   hmmm, something is not working as expected then, I have indexed more than Integer.MAX_VALUE documents and force merge to 1 segment and I get the following error:
   
   ```
   invalid softDelCount=0 (maxDoc=-2147482649)
   java.lang.IllegalArgumentException: invalid softDelCount=0 (maxDoc=-2147482649)
   	at org.apache.lucene.index.SegmentCommitInfo.setSoftDelCount(SegmentCommitInfo.java:387)
   	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5638)
   	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:5126)
   	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6932)
   	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:661)
   	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:726)
   ``` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6266512478

   Did a second pass to catch the other counts that were still int but can span the whole index now. Moved these to long: `IndexSearcher.count`, `TotalHitCountCollector` and its manager, `TopDocsCollector.totalHits` (it overflowed before being handed to the already-long `TotalHits`), and `CheckIndex`'s `totLoseDocCount`. The per-leaf `Weight#count` stays int.
   
   Some counts stay int on purpose because the consumer really needs one. Those spots fail fast instead of truncating quietly.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on PR #16847:
URL: https://github.com/apache/lucene/pull/16847#issuecomment-6267901461

   Is it possible to keep `int LeafReader.maxDoc()`? I guess that would require breaking the inheritance from `IndexReader`, but maybe it would be less disruptive, at least in terms of files/lines changed, than adding the new `leafMaxDoc`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Support more than 2^31 documents per index (long global doc ids) [lucene]

Posted by "iverase (via GitHub)" <gi...@apache.org>.
iverase commented on code in PR #16847:
URL: https://github.com/apache/lucene/pull/16847#discussion_r4311883430


##########
lucene/core/src/java/org/apache/lucene/index/IndexReader.java:
##########
@@ -294,22 +294,25 @@ public final int hashCode() {
    *
    * <p><b>NOTE</b>: This operation may run in O(maxDoc). Implementations that can't return this
    * number in constant-time should cache it.
+   *
+   * <p>Access the per-segment {@code int} count with {@link LeafReader#numLeafDocs()}.
    */
-  public abstract int numDocs();
+  public abstract long numDocs();

Review Comment:
   Do we need to change the methods #docFreq(Term term) and #getDocCount(String field) to return a long?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org