You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by "Kevin Liang (BLOOMBERG/ 919 3RD A)" <kl...@bloomberg.net> on 2025/07/23 15:43:35 UTC
Remaining work for Lucene 10
I see there's an open PR for going to Lucene 10.2 on solr main (https://github.com/apache/solr/pull/3053). I'm curious what the current status is (as in how much work remains on this)? And how best can I or others help on this work?
-Kevin Liang
Re: Remaining work for Lucene 10
Posted by David Smiley <ds...@apache.org>.
Ah; okay.
BTW, those are some excellent commit comments Ishan. Thanks for putting
the care and attention into them.
On Sat, Aug 9, 2025 at 6:33 PM Mark Miller <ma...@gmail.com> wrote:
> Because I was looking at some fails, like the grouping tests, and I didn't
> see a Lucene issue that should have broken them, but it is extremely
> difficult to work out an 8 month old branch with multiple non rebase merge
> ups starting from zero history working with the branch.
>
> There are also subjective choices you can make to do the upgrade, and I
> wanted to see what you actually had to drop or what tests had to be ignored
> if you just did a minimum job following Lucene’s Migrate document and
> CHANGES.txt.
>
> I didn't end up ignoring any tests, and I didn't drop preanalyzed field or
> Lucene native timeouts for example. That is much easier to do when you are
> familiar with all of the changes and work vs trying to jump in on a lot of
> work at the end of it with little context or experience with what all of
> the changes are.
>
> And now that you have merged up, I can easily diff the two branches and
> compare.
>
Re: Remaining work for Lucene 10
Posted by Mark Miller <ma...@gmail.com>.
Because I was looking at some fails, like the grouping tests, and I didn't
see a Lucene issue that should have broken them, but it is extremely
difficult to work out an 8 month old branch with multiple non rebase merge
ups starting from zero history working with the branch.
There are also subjective choices you can make to do the upgrade, and I
wanted to see what you actually had to drop or what tests had to be ignored
if you just did a minimum job following Lucene’s Migrate document and
CHANGES.txt.
I didn't end up ignoring any tests, and I didn't drop preanalyzed field or
Lucene native timeouts for example. That is much easier to do when you are
familiar with all of the changes and work vs trying to jump in on a lot of
work at the end of it with little context or experience with what all of
the changes are.
And now that you have merged up, I can easily diff the two branches and
compare.
Re: Remaining work for Lucene 10
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Hi Mark,
That statement represents my understanding at the time, when I was fighting
with debugging the issue.
Fixed it finally much more simply:
https://github.com/markrmiller/solr/commit/3f2503b672aa80f2b967f9f4a9d1d851b601c336
Sorry for the confusion.
Thanks and regards,
Ishan
On Sat, 9 Aug 2025 at 05:03, David Smiley <ds...@apache.org> wrote:
> Mark, why did you start an upgrade from scratch when there was already work
> in progress?
>
> (BTW I "watch" your fork and observed an upgrade guide, curiously,
> wondering how your effort relates to that of the community)
>
> On Fri, Aug 8, 2025 at 5:47 PM Mark Miller <ma...@gmail.com> wrote:
>
> > “Grouping behaviour has changed in Lucene in terms of how the order of
> > groups are returned. We either need to work around that to provide
> backward
> > compatibility or support the new behaviour. Christine has done some
> > investigation around it and I have been looking at it, on and off, for a
> > few months now. I'll update the PR to reflect this soon. There are some
> > other issues too, but this grouping thing is the major one sticking out.“
> >
> > I'm confused by this. Do you know what issue it happened in? I don't see
> > anything mentioned in the migration guide or changes from Lucene that
> > mentions this. I also did a quick Lucene 10.2.2 upgrade from scratch on
> > Wednesday, and I didn't have to touch anything in grouping to have all
> > tests pass.
> >
> > That's not to say you are wrong, but it does make me interested to know
> > what Lucene issue you are referring to that made this change.
> >
> > - MRM
> >
> >
> > On Thu, Aug 7, 2025 at 11:11 AM Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> > > I've merged SOLR-17631. Please review the changes on the PR #3053 and
> the
> > > now merged commits in the main branch.
> > >
> > > On Thu, 24 Jul 2025 at 14:24, Ishan Chattopadhyaya <
> > > ichattopadhyaya@gmail.com> wrote:
> > >
> > > > Grouping behaviour has changed in Lucene in terms of how the order of
> > > > groups are returned. We either need to work around that to provide
> > > backward
> > > > compatibility or support the new behaviour. Christine has done some
> > > > investigation around it and I have been looking at it, on and off,
> for
> > a
> > > > few months now. I'll update the PR to reflect this soon. There are
> some
> > > > other issues too, but this grouping thing is the major one sticking
> > out.
> > > >
> > > > On Wed, 23 Jul 2025 at 22:04, David Smiley <ds...@apache.org>
> wrote:
> > > >
> > > >> Commenting on the PR would be the place to offer to help, as the
> > > pertinent
> > > >> people involved would see it.
> > > >>
> > > >> On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A)
> <
> > > >> kliang78@bloomberg.net> wrote:
> > > >>
> > > >> > I see there's an open PR for going to Lucene 10.2 on solr main (
> > > >> > https://github.com/apache/solr/pull/3053). I'm curious what the
> > > current
> > > >> > status is (as in how much work remains on this)? And how best can
> I
> > or
> > > >> > others help on this work?
> > > >> >
> > > >> > -Kevin Liang
> > > >>
> > > >
> > >
> >
>
Re: Remaining work for Lucene 10
Posted by David Smiley <ds...@apache.org>.
Mark, why did you start an upgrade from scratch when there was already work
in progress?
(BTW I "watch" your fork and observed an upgrade guide, curiously,
wondering how your effort relates to that of the community)
On Fri, Aug 8, 2025 at 5:47 PM Mark Miller <ma...@gmail.com> wrote:
> “Grouping behaviour has changed in Lucene in terms of how the order of
> groups are returned. We either need to work around that to provide backward
> compatibility or support the new behaviour. Christine has done some
> investigation around it and I have been looking at it, on and off, for a
> few months now. I'll update the PR to reflect this soon. There are some
> other issues too, but this grouping thing is the major one sticking out.“
>
> I'm confused by this. Do you know what issue it happened in? I don't see
> anything mentioned in the migration guide or changes from Lucene that
> mentions this. I also did a quick Lucene 10.2.2 upgrade from scratch on
> Wednesday, and I didn't have to touch anything in grouping to have all
> tests pass.
>
> That's not to say you are wrong, but it does make me interested to know
> what Lucene issue you are referring to that made this change.
>
> - MRM
>
>
> On Thu, Aug 7, 2025 at 11:11 AM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > I've merged SOLR-17631. Please review the changes on the PR #3053 and the
> > now merged commits in the main branch.
> >
> > On Thu, 24 Jul 2025 at 14:24, Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> > > Grouping behaviour has changed in Lucene in terms of how the order of
> > > groups are returned. We either need to work around that to provide
> > backward
> > > compatibility or support the new behaviour. Christine has done some
> > > investigation around it and I have been looking at it, on and off, for
> a
> > > few months now. I'll update the PR to reflect this soon. There are some
> > > other issues too, but this grouping thing is the major one sticking
> out.
> > >
> > > On Wed, 23 Jul 2025 at 22:04, David Smiley <ds...@apache.org> wrote:
> > >
> > >> Commenting on the PR would be the place to offer to help, as the
> > pertinent
> > >> people involved would see it.
> > >>
> > >> On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A) <
> > >> kliang78@bloomberg.net> wrote:
> > >>
> > >> > I see there's an open PR for going to Lucene 10.2 on solr main (
> > >> > https://github.com/apache/solr/pull/3053). I'm curious what the
> > current
> > >> > status is (as in how much work remains on this)? And how best can I
> or
> > >> > others help on this work?
> > >> >
> > >> > -Kevin Liang
> > >>
> > >
> >
>
Re: Remaining work for Lucene 10
Posted by Mark Miller <ma...@gmail.com>.
“Grouping behaviour has changed in Lucene in terms of how the order of
groups are returned. We either need to work around that to provide backward
compatibility or support the new behaviour. Christine has done some
investigation around it and I have been looking at it, on and off, for a
few months now. I'll update the PR to reflect this soon. There are some
other issues too, but this grouping thing is the major one sticking out.“
I'm confused by this. Do you know what issue it happened in? I don't see
anything mentioned in the migration guide or changes from Lucene that
mentions this. I also did a quick Lucene 10.2.2 upgrade from scratch on
Wednesday, and I didn't have to touch anything in grouping to have all
tests pass.
That's not to say you are wrong, but it does make me interested to know
what Lucene issue you are referring to that made this change.
- MRM
On Thu, Aug 7, 2025 at 11:11 AM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> I've merged SOLR-17631. Please review the changes on the PR #3053 and the
> now merged commits in the main branch.
>
> On Thu, 24 Jul 2025 at 14:24, Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > Grouping behaviour has changed in Lucene in terms of how the order of
> > groups are returned. We either need to work around that to provide
> backward
> > compatibility or support the new behaviour. Christine has done some
> > investigation around it and I have been looking at it, on and off, for a
> > few months now. I'll update the PR to reflect this soon. There are some
> > other issues too, but this grouping thing is the major one sticking out.
> >
> > On Wed, 23 Jul 2025 at 22:04, David Smiley <ds...@apache.org> wrote:
> >
> >> Commenting on the PR would be the place to offer to help, as the
> pertinent
> >> people involved would see it.
> >>
> >> On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A) <
> >> kliang78@bloomberg.net> wrote:
> >>
> >> > I see there's an open PR for going to Lucene 10.2 on solr main (
> >> > https://github.com/apache/solr/pull/3053). I'm curious what the
> current
> >> > status is (as in how much work remains on this)? And how best can I or
> >> > others help on this work?
> >> >
> >> > -Kevin Liang
> >>
> >
>
Re: Remaining work for Lucene 10
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
I've merged SOLR-17631. Please review the changes on the PR #3053 and the
now merged commits in the main branch.
On Thu, 24 Jul 2025 at 14:24, Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> Grouping behaviour has changed in Lucene in terms of how the order of
> groups are returned. We either need to work around that to provide backward
> compatibility or support the new behaviour. Christine has done some
> investigation around it and I have been looking at it, on and off, for a
> few months now. I'll update the PR to reflect this soon. There are some
> other issues too, but this grouping thing is the major one sticking out.
>
> On Wed, 23 Jul 2025 at 22:04, David Smiley <ds...@apache.org> wrote:
>
>> Commenting on the PR would be the place to offer to help, as the pertinent
>> people involved would see it.
>>
>> On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A) <
>> kliang78@bloomberg.net> wrote:
>>
>> > I see there's an open PR for going to Lucene 10.2 on solr main (
>> > https://github.com/apache/solr/pull/3053). I'm curious what the current
>> > status is (as in how much work remains on this)? And how best can I or
>> > others help on this work?
>> >
>> > -Kevin Liang
>>
>
Re: Remaining work for Lucene 10
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Grouping behaviour has changed in Lucene in terms of how the order of
groups are returned. We either need to work around that to provide backward
compatibility or support the new behaviour. Christine has done some
investigation around it and I have been looking at it, on and off, for a
few months now. I'll update the PR to reflect this soon. There are some
other issues too, but this grouping thing is the major one sticking out.
On Wed, 23 Jul 2025 at 22:04, David Smiley <ds...@apache.org> wrote:
> Commenting on the PR would be the place to offer to help, as the pertinent
> people involved would see it.
>
> On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A) <
> kliang78@bloomberg.net> wrote:
>
> > I see there's an open PR for going to Lucene 10.2 on solr main (
> > https://github.com/apache/solr/pull/3053). I'm curious what the current
> > status is (as in how much work remains on this)? And how best can I or
> > others help on this work?
> >
> > -Kevin Liang
>
Re: Remaining work for Lucene 10
Posted by David Smiley <ds...@apache.org>.
Commenting on the PR would be the place to offer to help, as the pertinent
people involved would see it.
On Wed, Jul 23, 2025 at 11:43 AM Kevin Liang (BLOOMBERG/ 919 3RD A) <
kliang78@bloomberg.net> wrote:
> I see there's an open PR for going to Lucene 10.2 on solr main (
> https://github.com/apache/solr/pull/3053). I'm curious what the current
> status is (as in how much work remains on this)? And how best can I or
> others help on this work?
>
> -Kevin Liang