You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by David Smiley <ds...@apache.org> on 2026/07/02 16:20:18 UTC
State of Lucene Index Sorting support in Solr
I started investigating the current state of Lucene's index sorting support
in Solr. I had Claude Opus write a report for me. Rather than hoard it to
myself, I'm sharing with everyone in case others are wondering what's up as
well.
Background
----------
Lucene has supported index-level sorting since LUCENE-6766 (Lucene 6.2),
where segments are internally sorted by a configurable field order at
flush/merge time. This enables significant query-time optimizations --
when the query's sort matches the index sort, Lucene can skip entire
segments or terminate collection early.
Current Solr Support
--------------------
Solr does support index sorting today, but through an indirect mechanism:
Configuration is done via SortingMergePolicyFactory in solrconfig.xml:
<mergePolicyFactory
class="org.apache.solr.index.SortingMergePolicyFactory">
<str name="sort">timestamp desc</str>
<str name="wrapped.prefix">inner</str>
<str
name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
</mergePolicyFactory>
Internally, SortingMergePolicy is a FilterMergePolicy that does nothing
merge-policy-related -- it simply holds a Sort object. SolrIndexConfig
then has a special instanceof check that extracts this Sort and calls
IndexWriterConfig.setIndexSort(). The class itself has a TODO comment
acknowledging this is a workaround: "remove this and add indexSort
specification directly to solrconfig.xml?"
Query-side integration exists via the "segmentTerminateEarly" query
parameter, which wraps the collector in an EarlyTerminatingSortingCollector.
Note that this collector is @Deprecated -- modern Lucene's TopFieldCollector
handles early termination natively when it detects sorted segments.
The /admin/segments API (with coreInfo=true) exposes the indexSort
configuration and per-segment sort info.
AtomicUpdateDocumentMerger correctly detects fields used for index sorting
and prevents DocValues-only updates on them (a Lucene limitation).
Open Issues
-----------
Several open JIRA issues relate to this area.
SOLR-9108: Improve how index time sorting is configured
https://issues.apache.org/jira/browse/SOLR-9108
Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes
configuring index sort directly in solrconfig.xml alongside other
IndexWriter settings rather than piggybacking on the merge policy.
SOLR-13681: Make Lucene's index sorting directly configurable in Solr
https://issues.apache.org/jira/browse/SOLR-13681
Filed by Christine Poerschke in 2019. Has a draft PR (#313) that adds
a direct <indexSort> config element to solrconfig.xml. The PR has been
stalled since 2021; the main open question is what should happen when
index sorting is enabled on an existing collection that already has
unsorted segments. Duplicates SOLR-12230 (deprecate SortingMergePolicy).
SOLR-12239: Enabling index sorting causes CorruptIndexException
https://issues.apache.org/jira/browse/SOLR-12239
When index sorting is enabled on an existing collection with unsorted
segments, reloading throws: "segment not sorted with indexSort=null".
The current workaround is to delete all data and reindex from scratch.
Notably, the related LUCENE-9484 ("Allow index sorting to happen after
the fact") was fixed in Lucene 9.0, which allows merging unsorted
segments into sorted ones retroactively. Solr has not wired this up.
SOLR-17170: Support Blocks in Index Sorting
https://issues.apache.org/jira/browse/SOLR-17170
Lucene 9.10+ supports block-aware presort during index sorting (via
Lucene PR #12829). This is critical for nested/block-join documents.
No Solr-side work has been done.
Summary
-------
Index sorting in Solr works for simple (non-nested) use cases via the
SortingMergePolicyFactory, but the implementation is showing its age:
- Configuration is indirect and hacky (merge policy as a Sort carrier)
- Cannot be safely enabled on existing collections without full reindex,
despite Lucene having solved this at the engine level since 9.0
- Incompatible with nested/block-join documents on Solr 10+
- The query-side early termination collector is deprecated
- A draft PR for direct configuration has been stalled since 2021
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
Re: State of Lucene Index Sorting support in Solr
Posted by Serhiy Bzhezytskyy <me...@serhiy-bzhezytskyy.com>.
Thanks David for the feedback — it really charged my enthusiasm and keeps
the garage door open for more and more improvements (hope not breaking
things 🙂 ).
On the pace — the quick reviews were the major fuel for me during
onboarding, more than I expected. Now I'm ready to adapt to any pace that
works.
PR "index-sorting-all-in-one": github.com/apache/solr/pull/4648
On Tue, 21 Jul 2026 at 07:05, David Smiley <ds...@apache.org> wrote:
> I don't know where you came from, Serhiy, but I'm super appreciative of
> your efforts to improve Solr!
> BTW I hope you can get enough code reviews but that's a limiting factor
> that will require some patience.
> Personally I need to dedicate more time to my day job than in some previous
> weeks.
>
> On Fri, Jul 17, 2026 at 11:00 AM Serhiy Bzhezytskyy <
> me@serhiy-bzhezytskyy.com> wrote:
>
> > Hello All,
> > Following up on David's "State of Lucene Index Sorting" report from a few
> > weeks back. That report reads as a roadmap — a handful of related tickets
> > that together would make index sorting first-class in Solr — so I spent
> > time working through them end to end. I have local, tested
> implementations
> > for the open pieces and I'd like to bring them to the list before opening
> > PRs, since several sit in areas others own (particularly Christine's
> #313).
> >
> > I'll lay out how I see the puzzle, what I've built, and where I'd like
> > direction / to collaborate.
> >
> > *The picture (from David's report)*
> >
> > Index sorting works today, but only through the indirect
> > `SortingMergePolicyFactory` — a merge policy that does no merging and
> > exists only to carry a `Sort` to `IndexWriterConfig.setIndexSort()`.
> >
> > The report flagged five gaps; working through them surfaced two adjacent
> > pieces (SOLR-15390, SOLR-17310).
> >
> > The full map as I now see it:
> > - SOLR-13681 — direct <indexSort> config — Open, Christine's draft #313
> > - SOLR-9108 — improve sort config (overlaps 13681) — Open
> > - SOLR-12230 — deprecate SortingMergePolicy — Open
> > - SOLR-12239 — re-sort an existing collection — Open, my PR #4644
> > - SOLR-17170 — nested/block-join sorting — Resolved (David)
> > - SOLR-15390 — modernize the early-termination collector (adjacent) —
> Patch
> > Available
> > - SOLR-17310 — configurable leaf sorter (adjacent) — Open, Wei Wang's
> #2477
> > (stale-bot auto-closed, not rejected)
> >
> > *What I've implemented and verified locally*
> >
> > 1. SOLR-13681 — direct <indexSort> config.
> > A first-class <indexConfig><indexSort>timestamp
> > desc</indexSort></indexConfig>, parsed with the same SortSpecParsing the
> > query sort uses, replacing the merge-policy indirection. This
> reimplements
> > Christine's #313 approach on current main (the draft predates `MapWriter`
> > and David's SOLR-17170 parent-field work, so it needed reworking) and
> > composes with the nested-docs parent field. Christine — this is your lane
> > and I'm not looking to step on #313. I'd rather hand this over or work
> with
> > you on it — or simply absorb any of it into #313 as you see fit, with or
> > without me. I just found it was the keystone the other pieces depend on,
> > and its long-standing blocker ("what about existing collections?") is
> > answered by SOLR-12239.
> >
> > 2. SOLR-12239 — re-sort an existing collection (PR #4644, already up).
> > A RESORTINDEX core-admin action (v1 + v2) using LUCENE-9484
> > (SortingCodecReader + addIndexes) to sort pre-existing segments without
> > a full reindex — removing the "sort is immutable, must reindex"
> limitation.
> > It reads the configured <indexSort> as its target, so the workflow is:
> set
> > <indexSort> → run RESORTINDEX once.
> >
> > 3. SOLR-17310 — configurable leaf sorter (<segmentSort>).
> > Builds on Wei Wang's #2477 (credited). Exposes Lucene's `setLeafSorter`
> for
> > *between-segment* ordering, distinct from index sort's *within-segment*
> > order. Generalized beyond the time-only version to also accept a
> > numeric-field spec, since setLeafSorter takes any Comparator<LeafReader>.
> >
> > 4. SOLR-15390 — modernize the deprecated collector.
> > Solr's segmentTerminateEarly rides a
> > @Deprecated EarlyTerminatingSortingCollector; I swapped it for Lucene's
> > native TopFieldCollector.isEarlyTerminated(), preserving the
> > `segmentTerminatedEarly` response semantics. The ticket's broader ask is
> > deprecating the segmentTerminateEarly param in favor of `minExactCount`
> — I
> > did the internal modernization but left the param-deprecation as a
> separate
> > decision, since it's user-facing.
> >
> > 5. SOLR-12230 — deprecate SortingMergePolicy(Factory).
> > Once <indexSort> exists as the replacement, the old carrier can be
> > deprecated. Done as the last step, with a Solr-11 upgrade note.
> >
> > Everything is tested (unit + SolrCloud; ~40 tests) and passes errorprone
> /
> > check / ref-guide link checks.
> >
> > *Two design questions I'd like the list's view on*
> >
> > - Config shape: <indexSort> (within-segment) and <segmentSort>
> > (between-segment) as *separate* <indexConfig> elements, vs. the
> > <indexSorters> unified-element idea from SOLR-17310. My reading
> > of precedent leans separate — Lucene keeps `setIndexSort`/`setLeafSorter`
> > as distinct setters, and ES/OpenSearch expose only the within-segment
> axis
> > (index.sort.*) with no leaf-sorter equivalent — but I don't feel strongly
> > and want to hear opinions.
> > - segmentTerminateEarly vs minExactCount: actually deprecate the param
> > (SOLR-15390's literal ask), or just modernize the internals as I've done?
> >
> > What I'm asking is mainly direction and collaboration — especially with
> > Christine on 13681, since that's the keystone and her active work. If
> > there's interest, I'll open the pieces as separate per-ticket PRs (#4644
> > is already up). OR open PR with all changes to see full picture. Happy to
> > adjust any of it.
> >
> > Thanks,
> > Serhiy
> >
> > On Wed, 15 Jul 2026 at 20:01, David Smiley <ds...@apache.org> wrote:
> >
> > > I'm working on a project / idea that will require it. I'll share more
> > > about that later when appropriate, but it's too early now.
> > >
> > > At least the current state isn't bad. It works, especially with
> expanded
> > > functionality extending to schemas/use-cases with nested docs:
> SOLR-17170
> > > -- fully merged back to 9.x. There's a usability issue that I hope
> > > Christine will prioritize improving.
> > >
> > > I've heard Adrien and others extol the virtues/benefits of index
> sorting
> > > but haven't yet had deployed it to share. I will when I can.
> > >
> > > On Wed, Jul 15, 2026 at 8:06 AM Jason Gerlowski <gerlowskija@gmail.com
> >
> > > wrote:
> > >
> > > > Curious David - are you considering using index sorting for a
> > > > use-case, or have any experience with it to share in terms of
> > > > speedups, etc? Or more just something that struck your fancy?
> > > >
> > > > I remember index-sorting being something that came up in a Community
> > > > over Code "Birds of a Feather" session (in, I believe, Halifax?), as
> > > > one of the things Lucene supported that Solr should really consider
> > > > enabling out of the box for performance benefits. But I can't
> > > > remember the details of that discussion very well...
> > > >
> > > > Best,
> > > >
> > > > Jason
> > > >
> > > > On Thu, Jul 2, 2026 at 12:20 PM David Smiley <ds...@apache.org>
> > wrote:
> > > > >
> > > > > I started investigating the current state of Lucene's index sorting
> > > > support
> > > > > in Solr. I had Claude Opus write a report for me. Rather than
> hoard
> > > it
> > > > to
> > > > > myself, I'm sharing with everyone in case others are wondering
> what's
> > > up
> > > > as
> > > > > well.
> > > > >
> > > > >
> > > > > Background
> > > > > ----------
> > > > >
> > > > > Lucene has supported index-level sorting since LUCENE-6766 (Lucene
> > > 6.2),
> > > > > where segments are internally sorted by a configurable field order
> at
> > > > > flush/merge time. This enables significant query-time optimizations
> > --
> > > > > when the query's sort matches the index sort, Lucene can skip
> entire
> > > > > segments or terminate collection early.
> > > > >
> > > > >
> > > > > Current Solr Support
> > > > > --------------------
> > > > >
> > > > > Solr does support index sorting today, but through an indirect
> > > mechanism:
> > > > >
> > > > > Configuration is done via SortingMergePolicyFactory in
> > solrconfig.xml:
> > > > >
> > > > > <mergePolicyFactory
> > > > > class="org.apache.solr.index.SortingMergePolicyFactory">
> > > > > <str name="sort">timestamp desc</str>
> > > > > <str name="wrapped.prefix">inner</str>
> > > > > <str
> > > > >
> > name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
> > > > > </mergePolicyFactory>
> > > > >
> > > > > Internally, SortingMergePolicy is a FilterMergePolicy that does
> > nothing
> > > > > merge-policy-related -- it simply holds a Sort object.
> > SolrIndexConfig
> > > > > then has a special instanceof check that extracts this Sort and
> calls
> > > > > IndexWriterConfig.setIndexSort(). The class itself has a TODO
> comment
> > > > > acknowledging this is a workaround: "remove this and add indexSort
> > > > > specification directly to solrconfig.xml?"
> > > > >
> > > > > Query-side integration exists via the "segmentTerminateEarly" query
> > > > > parameter, which wraps the collector in an
> > > > EarlyTerminatingSortingCollector.
> > > > > Note that this collector is @Deprecated -- modern Lucene's
> > > > TopFieldCollector
> > > > > handles early termination natively when it detects sorted segments.
> > > > >
> > > > > The /admin/segments API (with coreInfo=true) exposes the indexSort
> > > > > configuration and per-segment sort info.
> > > > >
> > > > > AtomicUpdateDocumentMerger correctly detects fields used for index
> > > > sorting
> > > > > and prevents DocValues-only updates on them (a Lucene limitation).
> > > > >
> > > > >
> > > > > Open Issues
> > > > > -----------
> > > > >
> > > > > Several open JIRA issues relate to this area.
> > > > >
> > > > > SOLR-9108: Improve how index time sorting is configured
> > > > > https://issues.apache.org/jira/browse/SOLR-9108
> > > > >
> > > > > Filed by Mike McCandless in 2016 right after LUCENE-6766.
> Proposes
> > > > > configuring index sort directly in solrconfig.xml alongside other
> > > > > IndexWriter settings rather than piggybacking on the merge
> policy.
> > > > >
> > > > > SOLR-13681: Make Lucene's index sorting directly configurable in
> Solr
> > > > > https://issues.apache.org/jira/browse/SOLR-13681
> > > > >
> > > > > Filed by Christine Poerschke in 2019. Has a draft PR (#313) that
> > adds
> > > > > a direct <indexSort> config element to solrconfig.xml. The PR has
> > > been
> > > > > stalled since 2021; the main open question is what should happen
> > when
> > > > > index sorting is enabled on an existing collection that already
> has
> > > > > unsorted segments. Duplicates SOLR-12230 (deprecate
> > > > SortingMergePolicy).
> > > > >
> > > > > SOLR-12239: Enabling index sorting causes CorruptIndexException
> > > > > https://issues.apache.org/jira/browse/SOLR-12239
> > > > >
> > > > > When index sorting is enabled on an existing collection with
> > unsorted
> > > > > segments, reloading throws: "segment not sorted with
> > indexSort=null".
> > > > > The current workaround is to delete all data and reindex from
> > > scratch.
> > > > > Notably, the related LUCENE-9484 ("Allow index sorting to happen
> > > after
> > > > > the fact") was fixed in Lucene 9.0, which allows merging unsorted
> > > > > segments into sorted ones retroactively. Solr has not wired this
> > up.
> > > > >
> > > > > SOLR-17170: Support Blocks in Index Sorting
> > > > > https://issues.apache.org/jira/browse/SOLR-17170
> > > > >
> > > > > Lucene 9.10+ supports block-aware presort during index sorting
> (via
> > > > > Lucene PR #12829). This is critical for nested/block-join
> > documents.
> > > > > No Solr-side work has been done.
> > > > >
> > > > >
> > > > > Summary
> > > > > -------
> > > > >
> > > > > Index sorting in Solr works for simple (non-nested) use cases via
> the
> > > > > SortingMergePolicyFactory, but the implementation is showing its
> age:
> > > > >
> > > > > - Configuration is indirect and hacky (merge policy as a Sort
> > carrier)
> > > > > - Cannot be safely enabled on existing collections without full
> > > reindex,
> > > > > despite Lucene having solved this at the engine level since 9.0
> > > > > - Incompatible with nested/block-join documents on Solr 10+
> > > > > - The query-side early termination collector is deprecated
> > > > > - A draft PR for direct configuration has been stalled since 2021
> > > > >
> > > > > ~ David Smiley
> > > > > Apache Lucene/Solr Search Developer
> > > > > http://www.linkedin.com/in/davidwsmiley
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > > For additional commands, e-mail: dev-help@solr.apache.org
> > > >
> > > >
> > >
> >
>
Re: State of Lucene Index Sorting support in Solr
Posted by David Smiley <ds...@apache.org>.
I don't know where you came from, Serhiy, but I'm super appreciative of
your efforts to improve Solr!
BTW I hope you can get enough code reviews but that's a limiting factor
that will require some patience.
Personally I need to dedicate more time to my day job than in some previous
weeks.
On Fri, Jul 17, 2026 at 11:00 AM Serhiy Bzhezytskyy <
me@serhiy-bzhezytskyy.com> wrote:
> Hello All,
> Following up on David's "State of Lucene Index Sorting" report from a few
> weeks back. That report reads as a roadmap — a handful of related tickets
> that together would make index sorting first-class in Solr — so I spent
> time working through them end to end. I have local, tested implementations
> for the open pieces and I'd like to bring them to the list before opening
> PRs, since several sit in areas others own (particularly Christine's #313).
>
> I'll lay out how I see the puzzle, what I've built, and where I'd like
> direction / to collaborate.
>
> *The picture (from David's report)*
>
> Index sorting works today, but only through the indirect
> `SortingMergePolicyFactory` — a merge policy that does no merging and
> exists only to carry a `Sort` to `IndexWriterConfig.setIndexSort()`.
>
> The report flagged five gaps; working through them surfaced two adjacent
> pieces (SOLR-15390, SOLR-17310).
>
> The full map as I now see it:
> - SOLR-13681 — direct <indexSort> config — Open, Christine's draft #313
> - SOLR-9108 — improve sort config (overlaps 13681) — Open
> - SOLR-12230 — deprecate SortingMergePolicy — Open
> - SOLR-12239 — re-sort an existing collection — Open, my PR #4644
> - SOLR-17170 — nested/block-join sorting — Resolved (David)
> - SOLR-15390 — modernize the early-termination collector (adjacent) — Patch
> Available
> - SOLR-17310 — configurable leaf sorter (adjacent) — Open, Wei Wang's #2477
> (stale-bot auto-closed, not rejected)
>
> *What I've implemented and verified locally*
>
> 1. SOLR-13681 — direct <indexSort> config.
> A first-class <indexConfig><indexSort>timestamp
> desc</indexSort></indexConfig>, parsed with the same SortSpecParsing the
> query sort uses, replacing the merge-policy indirection. This reimplements
> Christine's #313 approach on current main (the draft predates `MapWriter`
> and David's SOLR-17170 parent-field work, so it needed reworking) and
> composes with the nested-docs parent field. Christine — this is your lane
> and I'm not looking to step on #313. I'd rather hand this over or work with
> you on it — or simply absorb any of it into #313 as you see fit, with or
> without me. I just found it was the keystone the other pieces depend on,
> and its long-standing blocker ("what about existing collections?") is
> answered by SOLR-12239.
>
> 2. SOLR-12239 — re-sort an existing collection (PR #4644, already up).
> A RESORTINDEX core-admin action (v1 + v2) using LUCENE-9484
> (SortingCodecReader + addIndexes) to sort pre-existing segments without
> a full reindex — removing the "sort is immutable, must reindex" limitation.
> It reads the configured <indexSort> as its target, so the workflow is: set
> <indexSort> → run RESORTINDEX once.
>
> 3. SOLR-17310 — configurable leaf sorter (<segmentSort>).
> Builds on Wei Wang's #2477 (credited). Exposes Lucene's `setLeafSorter` for
> *between-segment* ordering, distinct from index sort's *within-segment*
> order. Generalized beyond the time-only version to also accept a
> numeric-field spec, since setLeafSorter takes any Comparator<LeafReader>.
>
> 4. SOLR-15390 — modernize the deprecated collector.
> Solr's segmentTerminateEarly rides a
> @Deprecated EarlyTerminatingSortingCollector; I swapped it for Lucene's
> native TopFieldCollector.isEarlyTerminated(), preserving the
> `segmentTerminatedEarly` response semantics. The ticket's broader ask is
> deprecating the segmentTerminateEarly param in favor of `minExactCount` — I
> did the internal modernization but left the param-deprecation as a separate
> decision, since it's user-facing.
>
> 5. SOLR-12230 — deprecate SortingMergePolicy(Factory).
> Once <indexSort> exists as the replacement, the old carrier can be
> deprecated. Done as the last step, with a Solr-11 upgrade note.
>
> Everything is tested (unit + SolrCloud; ~40 tests) and passes errorprone /
> check / ref-guide link checks.
>
> *Two design questions I'd like the list's view on*
>
> - Config shape: <indexSort> (within-segment) and <segmentSort>
> (between-segment) as *separate* <indexConfig> elements, vs. the
> <indexSorters> unified-element idea from SOLR-17310. My reading
> of precedent leans separate — Lucene keeps `setIndexSort`/`setLeafSorter`
> as distinct setters, and ES/OpenSearch expose only the within-segment axis
> (index.sort.*) with no leaf-sorter equivalent — but I don't feel strongly
> and want to hear opinions.
> - segmentTerminateEarly vs minExactCount: actually deprecate the param
> (SOLR-15390's literal ask), or just modernize the internals as I've done?
>
> What I'm asking is mainly direction and collaboration — especially with
> Christine on 13681, since that's the keystone and her active work. If
> there's interest, I'll open the pieces as separate per-ticket PRs (#4644
> is already up). OR open PR with all changes to see full picture. Happy to
> adjust any of it.
>
> Thanks,
> Serhiy
>
> On Wed, 15 Jul 2026 at 20:01, David Smiley <ds...@apache.org> wrote:
>
> > I'm working on a project / idea that will require it. I'll share more
> > about that later when appropriate, but it's too early now.
> >
> > At least the current state isn't bad. It works, especially with expanded
> > functionality extending to schemas/use-cases with nested docs: SOLR-17170
> > -- fully merged back to 9.x. There's a usability issue that I hope
> > Christine will prioritize improving.
> >
> > I've heard Adrien and others extol the virtues/benefits of index sorting
> > but haven't yet had deployed it to share. I will when I can.
> >
> > On Wed, Jul 15, 2026 at 8:06 AM Jason Gerlowski <ge...@gmail.com>
> > wrote:
> >
> > > Curious David - are you considering using index sorting for a
> > > use-case, or have any experience with it to share in terms of
> > > speedups, etc? Or more just something that struck your fancy?
> > >
> > > I remember index-sorting being something that came up in a Community
> > > over Code "Birds of a Feather" session (in, I believe, Halifax?), as
> > > one of the things Lucene supported that Solr should really consider
> > > enabling out of the box for performance benefits. But I can't
> > > remember the details of that discussion very well...
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Thu, Jul 2, 2026 at 12:20 PM David Smiley <ds...@apache.org>
> wrote:
> > > >
> > > > I started investigating the current state of Lucene's index sorting
> > > support
> > > > in Solr. I had Claude Opus write a report for me. Rather than hoard
> > it
> > > to
> > > > myself, I'm sharing with everyone in case others are wondering what's
> > up
> > > as
> > > > well.
> > > >
> > > >
> > > > Background
> > > > ----------
> > > >
> > > > Lucene has supported index-level sorting since LUCENE-6766 (Lucene
> > 6.2),
> > > > where segments are internally sorted by a configurable field order at
> > > > flush/merge time. This enables significant query-time optimizations
> --
> > > > when the query's sort matches the index sort, Lucene can skip entire
> > > > segments or terminate collection early.
> > > >
> > > >
> > > > Current Solr Support
> > > > --------------------
> > > >
> > > > Solr does support index sorting today, but through an indirect
> > mechanism:
> > > >
> > > > Configuration is done via SortingMergePolicyFactory in
> solrconfig.xml:
> > > >
> > > > <mergePolicyFactory
> > > > class="org.apache.solr.index.SortingMergePolicyFactory">
> > > > <str name="sort">timestamp desc</str>
> > > > <str name="wrapped.prefix">inner</str>
> > > > <str
> > > >
> name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
> > > > </mergePolicyFactory>
> > > >
> > > > Internally, SortingMergePolicy is a FilterMergePolicy that does
> nothing
> > > > merge-policy-related -- it simply holds a Sort object.
> SolrIndexConfig
> > > > then has a special instanceof check that extracts this Sort and calls
> > > > IndexWriterConfig.setIndexSort(). The class itself has a TODO comment
> > > > acknowledging this is a workaround: "remove this and add indexSort
> > > > specification directly to solrconfig.xml?"
> > > >
> > > > Query-side integration exists via the "segmentTerminateEarly" query
> > > > parameter, which wraps the collector in an
> > > EarlyTerminatingSortingCollector.
> > > > Note that this collector is @Deprecated -- modern Lucene's
> > > TopFieldCollector
> > > > handles early termination natively when it detects sorted segments.
> > > >
> > > > The /admin/segments API (with coreInfo=true) exposes the indexSort
> > > > configuration and per-segment sort info.
> > > >
> > > > AtomicUpdateDocumentMerger correctly detects fields used for index
> > > sorting
> > > > and prevents DocValues-only updates on them (a Lucene limitation).
> > > >
> > > >
> > > > Open Issues
> > > > -----------
> > > >
> > > > Several open JIRA issues relate to this area.
> > > >
> > > > SOLR-9108: Improve how index time sorting is configured
> > > > https://issues.apache.org/jira/browse/SOLR-9108
> > > >
> > > > Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes
> > > > configuring index sort directly in solrconfig.xml alongside other
> > > > IndexWriter settings rather than piggybacking on the merge policy.
> > > >
> > > > SOLR-13681: Make Lucene's index sorting directly configurable in Solr
> > > > https://issues.apache.org/jira/browse/SOLR-13681
> > > >
> > > > Filed by Christine Poerschke in 2019. Has a draft PR (#313) that
> adds
> > > > a direct <indexSort> config element to solrconfig.xml. The PR has
> > been
> > > > stalled since 2021; the main open question is what should happen
> when
> > > > index sorting is enabled on an existing collection that already has
> > > > unsorted segments. Duplicates SOLR-12230 (deprecate
> > > SortingMergePolicy).
> > > >
> > > > SOLR-12239: Enabling index sorting causes CorruptIndexException
> > > > https://issues.apache.org/jira/browse/SOLR-12239
> > > >
> > > > When index sorting is enabled on an existing collection with
> unsorted
> > > > segments, reloading throws: "segment not sorted with
> indexSort=null".
> > > > The current workaround is to delete all data and reindex from
> > scratch.
> > > > Notably, the related LUCENE-9484 ("Allow index sorting to happen
> > after
> > > > the fact") was fixed in Lucene 9.0, which allows merging unsorted
> > > > segments into sorted ones retroactively. Solr has not wired this
> up.
> > > >
> > > > SOLR-17170: Support Blocks in Index Sorting
> > > > https://issues.apache.org/jira/browse/SOLR-17170
> > > >
> > > > Lucene 9.10+ supports block-aware presort during index sorting (via
> > > > Lucene PR #12829). This is critical for nested/block-join
> documents.
> > > > No Solr-side work has been done.
> > > >
> > > >
> > > > Summary
> > > > -------
> > > >
> > > > Index sorting in Solr works for simple (non-nested) use cases via the
> > > > SortingMergePolicyFactory, but the implementation is showing its age:
> > > >
> > > > - Configuration is indirect and hacky (merge policy as a Sort
> carrier)
> > > > - Cannot be safely enabled on existing collections without full
> > reindex,
> > > > despite Lucene having solved this at the engine level since 9.0
> > > > - Incompatible with nested/block-join documents on Solr 10+
> > > > - The query-side early termination collector is deprecated
> > > > - A draft PR for direct configuration has been stalled since 2021
> > > >
> > > > ~ David Smiley
> > > > Apache Lucene/Solr Search Developer
> > > > http://www.linkedin.com/in/davidwsmiley
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
> > >
> >
>
Re: State of Lucene Index Sorting support in Solr
Posted by Serhiy Bzhezytskyy <me...@serhiy-bzhezytskyy.com>.
Hello All,
Following up on David's "State of Lucene Index Sorting" report from a few
weeks back. That report reads as a roadmap — a handful of related tickets
that together would make index sorting first-class in Solr — so I spent
time working through them end to end. I have local, tested implementations
for the open pieces and I'd like to bring them to the list before opening
PRs, since several sit in areas others own (particularly Christine's #313).
I'll lay out how I see the puzzle, what I've built, and where I'd like
direction / to collaborate.
*The picture (from David's report)*
Index sorting works today, but only through the indirect
`SortingMergePolicyFactory` — a merge policy that does no merging and
exists only to carry a `Sort` to `IndexWriterConfig.setIndexSort()`.
The report flagged five gaps; working through them surfaced two adjacent
pieces (SOLR-15390, SOLR-17310).
The full map as I now see it:
- SOLR-13681 — direct <indexSort> config — Open, Christine's draft #313
- SOLR-9108 — improve sort config (overlaps 13681) — Open
- SOLR-12230 — deprecate SortingMergePolicy — Open
- SOLR-12239 — re-sort an existing collection — Open, my PR #4644
- SOLR-17170 — nested/block-join sorting — Resolved (David)
- SOLR-15390 — modernize the early-termination collector (adjacent) — Patch
Available
- SOLR-17310 — configurable leaf sorter (adjacent) — Open, Wei Wang's #2477
(stale-bot auto-closed, not rejected)
*What I've implemented and verified locally*
1. SOLR-13681 — direct <indexSort> config.
A first-class <indexConfig><indexSort>timestamp
desc</indexSort></indexConfig>, parsed with the same SortSpecParsing the
query sort uses, replacing the merge-policy indirection. This reimplements
Christine's #313 approach on current main (the draft predates `MapWriter`
and David's SOLR-17170 parent-field work, so it needed reworking) and
composes with the nested-docs parent field. Christine — this is your lane
and I'm not looking to step on #313. I'd rather hand this over or work with
you on it — or simply absorb any of it into #313 as you see fit, with or
without me. I just found it was the keystone the other pieces depend on,
and its long-standing blocker ("what about existing collections?") is
answered by SOLR-12239.
2. SOLR-12239 — re-sort an existing collection (PR #4644, already up).
A RESORTINDEX core-admin action (v1 + v2) using LUCENE-9484
(SortingCodecReader + addIndexes) to sort pre-existing segments without
a full reindex — removing the "sort is immutable, must reindex" limitation.
It reads the configured <indexSort> as its target, so the workflow is: set
<indexSort> → run RESORTINDEX once.
3. SOLR-17310 — configurable leaf sorter (<segmentSort>).
Builds on Wei Wang's #2477 (credited). Exposes Lucene's `setLeafSorter` for
*between-segment* ordering, distinct from index sort's *within-segment*
order. Generalized beyond the time-only version to also accept a
numeric-field spec, since setLeafSorter takes any Comparator<LeafReader>.
4. SOLR-15390 — modernize the deprecated collector.
Solr's segmentTerminateEarly rides a
@Deprecated EarlyTerminatingSortingCollector; I swapped it for Lucene's
native TopFieldCollector.isEarlyTerminated(), preserving the
`segmentTerminatedEarly` response semantics. The ticket's broader ask is
deprecating the segmentTerminateEarly param in favor of `minExactCount` — I
did the internal modernization but left the param-deprecation as a separate
decision, since it's user-facing.
5. SOLR-12230 — deprecate SortingMergePolicy(Factory).
Once <indexSort> exists as the replacement, the old carrier can be
deprecated. Done as the last step, with a Solr-11 upgrade note.
Everything is tested (unit + SolrCloud; ~40 tests) and passes errorprone /
check / ref-guide link checks.
*Two design questions I'd like the list's view on*
- Config shape: <indexSort> (within-segment) and <segmentSort>
(between-segment) as *separate* <indexConfig> elements, vs. the
<indexSorters> unified-element idea from SOLR-17310. My reading
of precedent leans separate — Lucene keeps `setIndexSort`/`setLeafSorter`
as distinct setters, and ES/OpenSearch expose only the within-segment axis
(index.sort.*) with no leaf-sorter equivalent — but I don't feel strongly
and want to hear opinions.
- segmentTerminateEarly vs minExactCount: actually deprecate the param
(SOLR-15390's literal ask), or just modernize the internals as I've done?
What I'm asking is mainly direction and collaboration — especially with
Christine on 13681, since that's the keystone and her active work. If
there's interest, I'll open the pieces as separate per-ticket PRs (#4644
is already up). OR open PR with all changes to see full picture. Happy to
adjust any of it.
Thanks,
Serhiy
On Wed, 15 Jul 2026 at 20:01, David Smiley <ds...@apache.org> wrote:
> I'm working on a project / idea that will require it. I'll share more
> about that later when appropriate, but it's too early now.
>
> At least the current state isn't bad. It works, especially with expanded
> functionality extending to schemas/use-cases with nested docs: SOLR-17170
> -- fully merged back to 9.x. There's a usability issue that I hope
> Christine will prioritize improving.
>
> I've heard Adrien and others extol the virtues/benefits of index sorting
> but haven't yet had deployed it to share. I will when I can.
>
> On Wed, Jul 15, 2026 at 8:06 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
>
> > Curious David - are you considering using index sorting for a
> > use-case, or have any experience with it to share in terms of
> > speedups, etc? Or more just something that struck your fancy?
> >
> > I remember index-sorting being something that came up in a Community
> > over Code "Birds of a Feather" session (in, I believe, Halifax?), as
> > one of the things Lucene supported that Solr should really consider
> > enabling out of the box for performance benefits. But I can't
> > remember the details of that discussion very well...
> >
> > Best,
> >
> > Jason
> >
> > On Thu, Jul 2, 2026 at 12:20 PM David Smiley <ds...@apache.org> wrote:
> > >
> > > I started investigating the current state of Lucene's index sorting
> > support
> > > in Solr. I had Claude Opus write a report for me. Rather than hoard
> it
> > to
> > > myself, I'm sharing with everyone in case others are wondering what's
> up
> > as
> > > well.
> > >
> > >
> > > Background
> > > ----------
> > >
> > > Lucene has supported index-level sorting since LUCENE-6766 (Lucene
> 6.2),
> > > where segments are internally sorted by a configurable field order at
> > > flush/merge time. This enables significant query-time optimizations --
> > > when the query's sort matches the index sort, Lucene can skip entire
> > > segments or terminate collection early.
> > >
> > >
> > > Current Solr Support
> > > --------------------
> > >
> > > Solr does support index sorting today, but through an indirect
> mechanism:
> > >
> > > Configuration is done via SortingMergePolicyFactory in solrconfig.xml:
> > >
> > > <mergePolicyFactory
> > > class="org.apache.solr.index.SortingMergePolicyFactory">
> > > <str name="sort">timestamp desc</str>
> > > <str name="wrapped.prefix">inner</str>
> > > <str
> > > name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
> > > </mergePolicyFactory>
> > >
> > > Internally, SortingMergePolicy is a FilterMergePolicy that does nothing
> > > merge-policy-related -- it simply holds a Sort object. SolrIndexConfig
> > > then has a special instanceof check that extracts this Sort and calls
> > > IndexWriterConfig.setIndexSort(). The class itself has a TODO comment
> > > acknowledging this is a workaround: "remove this and add indexSort
> > > specification directly to solrconfig.xml?"
> > >
> > > Query-side integration exists via the "segmentTerminateEarly" query
> > > parameter, which wraps the collector in an
> > EarlyTerminatingSortingCollector.
> > > Note that this collector is @Deprecated -- modern Lucene's
> > TopFieldCollector
> > > handles early termination natively when it detects sorted segments.
> > >
> > > The /admin/segments API (with coreInfo=true) exposes the indexSort
> > > configuration and per-segment sort info.
> > >
> > > AtomicUpdateDocumentMerger correctly detects fields used for index
> > sorting
> > > and prevents DocValues-only updates on them (a Lucene limitation).
> > >
> > >
> > > Open Issues
> > > -----------
> > >
> > > Several open JIRA issues relate to this area.
> > >
> > > SOLR-9108: Improve how index time sorting is configured
> > > https://issues.apache.org/jira/browse/SOLR-9108
> > >
> > > Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes
> > > configuring index sort directly in solrconfig.xml alongside other
> > > IndexWriter settings rather than piggybacking on the merge policy.
> > >
> > > SOLR-13681: Make Lucene's index sorting directly configurable in Solr
> > > https://issues.apache.org/jira/browse/SOLR-13681
> > >
> > > Filed by Christine Poerschke in 2019. Has a draft PR (#313) that adds
> > > a direct <indexSort> config element to solrconfig.xml. The PR has
> been
> > > stalled since 2021; the main open question is what should happen when
> > > index sorting is enabled on an existing collection that already has
> > > unsorted segments. Duplicates SOLR-12230 (deprecate
> > SortingMergePolicy).
> > >
> > > SOLR-12239: Enabling index sorting causes CorruptIndexException
> > > https://issues.apache.org/jira/browse/SOLR-12239
> > >
> > > When index sorting is enabled on an existing collection with unsorted
> > > segments, reloading throws: "segment not sorted with indexSort=null".
> > > The current workaround is to delete all data and reindex from
> scratch.
> > > Notably, the related LUCENE-9484 ("Allow index sorting to happen
> after
> > > the fact") was fixed in Lucene 9.0, which allows merging unsorted
> > > segments into sorted ones retroactively. Solr has not wired this up.
> > >
> > > SOLR-17170: Support Blocks in Index Sorting
> > > https://issues.apache.org/jira/browse/SOLR-17170
> > >
> > > Lucene 9.10+ supports block-aware presort during index sorting (via
> > > Lucene PR #12829). This is critical for nested/block-join documents.
> > > No Solr-side work has been done.
> > >
> > >
> > > Summary
> > > -------
> > >
> > > Index sorting in Solr works for simple (non-nested) use cases via the
> > > SortingMergePolicyFactory, but the implementation is showing its age:
> > >
> > > - Configuration is indirect and hacky (merge policy as a Sort carrier)
> > > - Cannot be safely enabled on existing collections without full
> reindex,
> > > despite Lucene having solved this at the engine level since 9.0
> > > - Incompatible with nested/block-join documents on Solr 10+
> > > - The query-side early termination collector is deprecated
> > > - A draft PR for direct configuration has been stalled since 2021
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
>
Re: State of Lucene Index Sorting support in Solr
Posted by David Smiley <ds...@apache.org>.
I'm working on a project / idea that will require it. I'll share more
about that later when appropriate, but it's too early now.
At least the current state isn't bad. It works, especially with expanded
functionality extending to schemas/use-cases with nested docs: SOLR-17170
-- fully merged back to 9.x. There's a usability issue that I hope
Christine will prioritize improving.
I've heard Adrien and others extol the virtues/benefits of index sorting
but haven't yet had deployed it to share. I will when I can.
On Wed, Jul 15, 2026 at 8:06 AM Jason Gerlowski <ge...@gmail.com>
wrote:
> Curious David - are you considering using index sorting for a
> use-case, or have any experience with it to share in terms of
> speedups, etc? Or more just something that struck your fancy?
>
> I remember index-sorting being something that came up in a Community
> over Code "Birds of a Feather" session (in, I believe, Halifax?), as
> one of the things Lucene supported that Solr should really consider
> enabling out of the box for performance benefits. But I can't
> remember the details of that discussion very well...
>
> Best,
>
> Jason
>
> On Thu, Jul 2, 2026 at 12:20 PM David Smiley <ds...@apache.org> wrote:
> >
> > I started investigating the current state of Lucene's index sorting
> support
> > in Solr. I had Claude Opus write a report for me. Rather than hoard it
> to
> > myself, I'm sharing with everyone in case others are wondering what's up
> as
> > well.
> >
> >
> > Background
> > ----------
> >
> > Lucene has supported index-level sorting since LUCENE-6766 (Lucene 6.2),
> > where segments are internally sorted by a configurable field order at
> > flush/merge time. This enables significant query-time optimizations --
> > when the query's sort matches the index sort, Lucene can skip entire
> > segments or terminate collection early.
> >
> >
> > Current Solr Support
> > --------------------
> >
> > Solr does support index sorting today, but through an indirect mechanism:
> >
> > Configuration is done via SortingMergePolicyFactory in solrconfig.xml:
> >
> > <mergePolicyFactory
> > class="org.apache.solr.index.SortingMergePolicyFactory">
> > <str name="sort">timestamp desc</str>
> > <str name="wrapped.prefix">inner</str>
> > <str
> > name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
> > </mergePolicyFactory>
> >
> > Internally, SortingMergePolicy is a FilterMergePolicy that does nothing
> > merge-policy-related -- it simply holds a Sort object. SolrIndexConfig
> > then has a special instanceof check that extracts this Sort and calls
> > IndexWriterConfig.setIndexSort(). The class itself has a TODO comment
> > acknowledging this is a workaround: "remove this and add indexSort
> > specification directly to solrconfig.xml?"
> >
> > Query-side integration exists via the "segmentTerminateEarly" query
> > parameter, which wraps the collector in an
> EarlyTerminatingSortingCollector.
> > Note that this collector is @Deprecated -- modern Lucene's
> TopFieldCollector
> > handles early termination natively when it detects sorted segments.
> >
> > The /admin/segments API (with coreInfo=true) exposes the indexSort
> > configuration and per-segment sort info.
> >
> > AtomicUpdateDocumentMerger correctly detects fields used for index
> sorting
> > and prevents DocValues-only updates on them (a Lucene limitation).
> >
> >
> > Open Issues
> > -----------
> >
> > Several open JIRA issues relate to this area.
> >
> > SOLR-9108: Improve how index time sorting is configured
> > https://issues.apache.org/jira/browse/SOLR-9108
> >
> > Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes
> > configuring index sort directly in solrconfig.xml alongside other
> > IndexWriter settings rather than piggybacking on the merge policy.
> >
> > SOLR-13681: Make Lucene's index sorting directly configurable in Solr
> > https://issues.apache.org/jira/browse/SOLR-13681
> >
> > Filed by Christine Poerschke in 2019. Has a draft PR (#313) that adds
> > a direct <indexSort> config element to solrconfig.xml. The PR has been
> > stalled since 2021; the main open question is what should happen when
> > index sorting is enabled on an existing collection that already has
> > unsorted segments. Duplicates SOLR-12230 (deprecate
> SortingMergePolicy).
> >
> > SOLR-12239: Enabling index sorting causes CorruptIndexException
> > https://issues.apache.org/jira/browse/SOLR-12239
> >
> > When index sorting is enabled on an existing collection with unsorted
> > segments, reloading throws: "segment not sorted with indexSort=null".
> > The current workaround is to delete all data and reindex from scratch.
> > Notably, the related LUCENE-9484 ("Allow index sorting to happen after
> > the fact") was fixed in Lucene 9.0, which allows merging unsorted
> > segments into sorted ones retroactively. Solr has not wired this up.
> >
> > SOLR-17170: Support Blocks in Index Sorting
> > https://issues.apache.org/jira/browse/SOLR-17170
> >
> > Lucene 9.10+ supports block-aware presort during index sorting (via
> > Lucene PR #12829). This is critical for nested/block-join documents.
> > No Solr-side work has been done.
> >
> >
> > Summary
> > -------
> >
> > Index sorting in Solr works for simple (non-nested) use cases via the
> > SortingMergePolicyFactory, but the implementation is showing its age:
> >
> > - Configuration is indirect and hacky (merge policy as a Sort carrier)
> > - Cannot be safely enabled on existing collections without full reindex,
> > despite Lucene having solved this at the engine level since 9.0
> > - Incompatible with nested/block-join documents on Solr 10+
> > - The query-side early termination collector is deprecated
> > - A draft PR for direct configuration has been stalled since 2021
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: State of Lucene Index Sorting support in Solr
Posted by Jason Gerlowski <ge...@gmail.com>.
Curious David - are you considering using index sorting for a
use-case, or have any experience with it to share in terms of
speedups, etc? Or more just something that struck your fancy?
I remember index-sorting being something that came up in a Community
over Code "Birds of a Feather" session (in, I believe, Halifax?), as
one of the things Lucene supported that Solr should really consider
enabling out of the box for performance benefits. But I can't
remember the details of that discussion very well...
Best,
Jason
On Thu, Jul 2, 2026 at 12:20 PM David Smiley <ds...@apache.org> wrote:
>
> I started investigating the current state of Lucene's index sorting support
> in Solr. I had Claude Opus write a report for me. Rather than hoard it to
> myself, I'm sharing with everyone in case others are wondering what's up as
> well.
>
>
> Background
> ----------
>
> Lucene has supported index-level sorting since LUCENE-6766 (Lucene 6.2),
> where segments are internally sorted by a configurable field order at
> flush/merge time. This enables significant query-time optimizations --
> when the query's sort matches the index sort, Lucene can skip entire
> segments or terminate collection early.
>
>
> Current Solr Support
> --------------------
>
> Solr does support index sorting today, but through an indirect mechanism:
>
> Configuration is done via SortingMergePolicyFactory in solrconfig.xml:
>
> <mergePolicyFactory
> class="org.apache.solr.index.SortingMergePolicyFactory">
> <str name="sort">timestamp desc</str>
> <str name="wrapped.prefix">inner</str>
> <str
> name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
> </mergePolicyFactory>
>
> Internally, SortingMergePolicy is a FilterMergePolicy that does nothing
> merge-policy-related -- it simply holds a Sort object. SolrIndexConfig
> then has a special instanceof check that extracts this Sort and calls
> IndexWriterConfig.setIndexSort(). The class itself has a TODO comment
> acknowledging this is a workaround: "remove this and add indexSort
> specification directly to solrconfig.xml?"
>
> Query-side integration exists via the "segmentTerminateEarly" query
> parameter, which wraps the collector in an EarlyTerminatingSortingCollector.
> Note that this collector is @Deprecated -- modern Lucene's TopFieldCollector
> handles early termination natively when it detects sorted segments.
>
> The /admin/segments API (with coreInfo=true) exposes the indexSort
> configuration and per-segment sort info.
>
> AtomicUpdateDocumentMerger correctly detects fields used for index sorting
> and prevents DocValues-only updates on them (a Lucene limitation).
>
>
> Open Issues
> -----------
>
> Several open JIRA issues relate to this area.
>
> SOLR-9108: Improve how index time sorting is configured
> https://issues.apache.org/jira/browse/SOLR-9108
>
> Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes
> configuring index sort directly in solrconfig.xml alongside other
> IndexWriter settings rather than piggybacking on the merge policy.
>
> SOLR-13681: Make Lucene's index sorting directly configurable in Solr
> https://issues.apache.org/jira/browse/SOLR-13681
>
> Filed by Christine Poerschke in 2019. Has a draft PR (#313) that adds
> a direct <indexSort> config element to solrconfig.xml. The PR has been
> stalled since 2021; the main open question is what should happen when
> index sorting is enabled on an existing collection that already has
> unsorted segments. Duplicates SOLR-12230 (deprecate SortingMergePolicy).
>
> SOLR-12239: Enabling index sorting causes CorruptIndexException
> https://issues.apache.org/jira/browse/SOLR-12239
>
> When index sorting is enabled on an existing collection with unsorted
> segments, reloading throws: "segment not sorted with indexSort=null".
> The current workaround is to delete all data and reindex from scratch.
> Notably, the related LUCENE-9484 ("Allow index sorting to happen after
> the fact") was fixed in Lucene 9.0, which allows merging unsorted
> segments into sorted ones retroactively. Solr has not wired this up.
>
> SOLR-17170: Support Blocks in Index Sorting
> https://issues.apache.org/jira/browse/SOLR-17170
>
> Lucene 9.10+ supports block-aware presort during index sorting (via
> Lucene PR #12829). This is critical for nested/block-join documents.
> No Solr-side work has been done.
>
>
> Summary
> -------
>
> Index sorting in Solr works for simple (non-nested) use cases via the
> SortingMergePolicyFactory, but the implementation is showing its age:
>
> - Configuration is indirect and hacky (merge policy as a Sort carrier)
> - Cannot be safely enabled on existing collections without full reindex,
> despite Lucene having solved this at the engine level since 9.0
> - Incompatible with nested/block-join documents on Solr 10+
> - The query-side early termination collector is deprecated
> - A draft PR for direct configuration has been stalled since 2021
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org