You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "serhiy-bzhezytskyy (via GitHub)" <gi...@apache.org> on 2026/07/21 06:12:12 UTC
[PR] Index Sorting: the full picture (SOLR-15390/13681/12239/17310/12230) — for review, happy to split [solr]
serhiy-bzhezytskyy opened a new pull request, #4648:
URL: https://github.com/apache/solr/pull/4648
Follows up on the dev@ thread ["State of Lucene Index Sorting support in Solr"](https://lists.apache.org/thread/qlzqjw4mlf84n90mxbbml35x866gfk3w).
**Purpose: this PR exists to make the *whole shape* of the index-sorting work reviewable in one place** — as I offered on the list ("open a PR with all changes to see the full picture"). It is **not** a request to merge all of this as one lump. I'm very happy to split it into per-ticket PRs, and to fold the config piece into Christine's existing work rather than land my own — see the note at the bottom.
## The picture (from David's "State of Lucene Index Sorting" report)
Index sorting works today only through the indirect `SortingMergePolicyFactory` (a merge policy that does no merging, existing only to carry a `Sort` to `IndexWriterConfig.setIndexSort()`). This branch works through the open pieces end to end, in dependency order:
| commit | ticket | what |
|--------|--------|------|
| SOLR-15390 | [15390](https://issues.apache.org/jira/browse/SOLR-15390) | use native `TopFieldCollector` early termination for `segmentTerminateEarly`; remove the deprecated `EarlyTerminatingSortingCollector` (Tomas said "feel free to take it") |
| SOLR-13681 | [13681](https://issues.apache.org/jira/browse/SOLR-13681) | direct `<indexSort>` config in `<indexConfig>`, replacing the merge-policy indirection |
| SOLR-12239 | [12239](https://issues.apache.org/jira/browse/SOLR-12239) | `RESORTINDEX` core-admin action to re-sort an existing index (also up standalone as #4644) |
| SOLR-17310 | [17310](https://issues.apache.org/jira/browse/SOLR-17310) | `<segmentSort>` to configure Lucene's leaf sorter |
| SOLR-12230 | [12230](https://issues.apache.org/jira/browse/SOLR-12230) | deprecate `SortingMergePolicy` now that `<indexSort>` replaces it |
## Credit (important — please read)
- **SOLR-13681** reimplements the approach from **@cpoerschke's draft #313** on current `main` (the draft predates `MapWriter` and some config changes). Christine owns this lane; I'd rather help land #313 than land a competing change — this is here for the full picture, not to bypass her work.
- **SOLR-17310** builds on **@weiwang19's #2477** (auto-closed as stale, not rejected). Credit for the leaf-sorter direction is his.
(I stripped `Co-authored-by` trailers from the commits deliberately — I didn't want to attribute code to Christine or Wei publicly without asking them first. Crediting them here in prose instead; happy to add trailers back if they'd prefer.)
## Open design questions (for the list / reviewers)
1. **Config shape:** `<indexSort>` (within-segment) and `<segmentSort>` (between-segment) as *separate* `<indexConfig>` elements, vs. a unified `<indexSorters>` element. My reading of precedent leans separate (Lucene keeps `setIndexSort`/`setLeafSorter` distinct; ES/OpenSearch expose only the within-segment axis) — but no strong opinion.
2. **segmentTerminateEarly:** actually deprecate the param (15390's literal ask), or just modernize internals? This branch modernizes internals; the param-deprecation is left as a separate user-facing decision.
## Testing
Unit + SolrCloud (~40 tests) pass locally, incl. `TestSegmentSorting` (`testSegmentTerminateEarly`, `testSegmentTerminateEarlyWithIndexSort`, `testAtomicUpdateOfSegmentSortField`) and `TestEarlyTerminatingQueries`. Compiles clean.
## What I'm asking
Direction and collaboration — especially with @cpoerschke on 13681, since that's the keystone and her active work. If the shape looks right, I'll break this into reviewable per-ticket PRs (15390 first — it's self-contained). No rush.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] Index Sorting: the full picture (SOLR-15390/13681/12239/17310/12230) — for review, happy to split [solr]
Posted by "serhiy-bzhezytskyy (via GitHub)" <gi...@apache.org>.
serhiy-bzhezytskyy closed pull request #4648: Index Sorting: the full picture (SOLR-15390/13681/12239/17310/12230) — for review, happy to split
URL: https://github.com/apache/solr/pull/4648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] Index Sorting: the full picture (SOLR-15390/13681/12239/17310/12230) — for review, happy to split [solr]
Posted by "serhiy-bzhezytskyy (via GitHub)" <gi...@apache.org>.
serhiy-bzhezytskyy commented on PR #4648:
URL: https://github.com/apache/solr/pull/4648#issuecomment-5476718520
Closing this. It was opened to make the whole shape of the index-sorting work reviewable in one place rather than to merge as one lump, and that job is done — the dev@ thread carries the shape, and the branch has since drifted (41 days, now conflicting).
Where the five pieces stand:
- **SOLR-15390** — split out as #4804, open for review: removes `EarlyTerminatingSortingCollector` in favour of native `TopFieldCollector` early termination.
- **SOLR-13681** `<indexSort>` — not mine to carry; the draft in #313 remains the reference.
- **SOLR-12239**, **SOLR-17310**, **SOLR-12230** — no PR from me. The branch `index-sorting-all-in-one` stays in place if anyone wants the commits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org