You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Noble Paul <no...@gmail.com> on 2021/05/04 04:31:52 UTC
Re: Separate git repo(s) for Solr modules
I'm still struggling to understand the workflow when I'm working on a
feature that spans lucene and solr.
I'm yet to see an argument against sub-modules
On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
wrote:
> > Shoving such a component into lucene-solr repo makes no sense, given its
> branching strategy is based on master / branch_8x
>
> I get how this could cause issues - I def hadn't thought much about
> multi-version support and branching. But I don't think moving plugins
> to a separate repo solves that problem for us. If our first class
> plugins are set up to have different release "lines" that don't line
> up with major Solr versions, it's only a matter of time before two of
> those plugins have branch requirements that conflict. Unless I'm
> missing something here?
>
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real. Doing it prematurely could lead to
> atrophy of the module....
>
> +1 to David's comment. I def hadn't considered the branching-scheme
> issues that Ishan brought up in his last reply, and they seem like
> valid concerns to me. But the risk and the downsides of "atrophy" are
> serious enough that I'd vote to not risk them until this starts to
> cause us issues in practice. Even if, for now, that means we won't be
> able to build a single plugin jar that supports (e.g.) 3 major Solr
> versions. IMO that's a much smaller loss.
>
> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org> wrote:
> >
> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
> epugh@opensourceconnections.com> wrote:
> >>
> >> Testing across multiple versions is always very difficult ;-). I
> recently saw this very interesting approach to using our Dockerized Solr’s
> to test a component against a number of previous versions of Solr.
> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
> model for other packages to adopt.
> >
> >
> > Thanks for the link to that Querqy PR. That is *very* similar to what I
> do at work (minus multi-version testing), and also similar to how I test
> multiple versions in one of my pet projects by using a CI build matrix of a
> configurable dependency. I didn't know Testcontainer.org has its own Solr
> module -- https://www.testcontainers.org/modules/solr/ but we implemented
> that ourselves; not hard.
> >
> >>
> >> Trying to maintain across multiple versions is kind of a Sisyphean
> task, and I don’t think plays to open source communities strengths. It’s
> hard enough to keep all components of Solr up to date with the latest
> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
> currently broken on master) . Reminds me of the Apache Gump project ;-)
> >>
> >> If something is really going to be backcompatible across certain
> versions, then maybe having it’s own repo makes sense,
> >
> >
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real. Doing it prematurely could lead to
> atrophy of the module....
> >
> >>
> >> but I suspect it means those components may go stale…. Look at DIH
> and Velocity components that are moved over to their own repos, both are
> getting stale, and I’d argue it’s because they don’t live in the main
> project where all of us have oversight and easy access.
> >
> >
> > Agreed :-(
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
-----------------------------------------------------
Noble Paul
Re: Separate git repo(s) for Solr modules
Posted by Houston Putman <ho...@gmail.com>.
I completely agree with Ilan.
Keeping the projects in the same repo was the only long term viable way of
having Solr depend on a snapshot version of Lucene.
And I'm sorry if I misspoke earlier. You can always "work on both at
the same time" even when we rely on a release version of Lucene,
but you won't be able to commit the Solr code until a Lucene release has
been made that includes those changes.
Also I think discussion has taken over the wrong thread. The thread was
originally about modules for various Solr plugins and projects.
- Houston
On Tue, May 4, 2021 at 7:02 AM Ilan Ginzburg <il...@gmail.com> wrote:
> As with any dependency on any project, you update the dependency project
> first then consume the updated dependency in Solr.
>
> If the idea is to be able to modify Lucene and Solr in parallel, then the
> project split is counterproductive.
>
> From the Solr perspective, Lucene and Zookerper are really two “similar”
> dependencies and IMO we should think about them in that way.
>
> Ilan
>
> On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
>
>> @Houston
>>
>> So, Are you suggesting we should not do that ?
>>
>> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com>
>> wrote:
>>
>>> In the future we wont be able to “work on both at the same time”, once
>>> Lucene 9 is cut. Why not pull that bandaid now?
>>>
>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>>>
>>>> I'm still struggling to understand the workflow when I'm working on a
>>>> feature that spans lucene and solr.
>>>>
>>>> I'm yet to see an argument against sub-modules
>>>>
>>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
>>>> wrote:
>>>>
>>>>> > Shoving such a component into lucene-solr repo makes no sense, given
>>>>> its branching strategy is based on master / branch_8x
>>>>>
>>>>> I get how this could cause issues - I def hadn't thought much about
>>>>> multi-version support and branching. But I don't think moving plugins
>>>>> to a separate repo solves that problem for us. If our first class
>>>>> plugins are set up to have different release "lines" that don't line
>>>>> up with major Solr versions, it's only a matter of time before two of
>>>>> those plugins have branch requirements that conflict. Unless I'm
>>>>> missing something here?
>>>>>
>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>> atrophy of the module....
>>>>>
>>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>>> issues that Ishan brought up in his last reply, and they seem like
>>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>>> serious enough that I'd vote to not risk them until this starts to
>>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>>> versions. IMO that's a much smaller loss.
>>>>>
>>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>>>> epugh@opensourceconnections.com> wrote:
>>>>> >>
>>>>> >> Testing across multiple versions is always very difficult ;-). I
>>>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>>>> to test a component against a number of previous versions of Solr.
>>>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>>>> model for other packages to adopt.
>>>>> >
>>>>> >
>>>>> > Thanks for the link to that Querqy PR. That is *very* similar to
>>>>> what I do at work (minus multi-version testing), and also similar to how I
>>>>> test multiple versions in one of my pet projects by using a CI build matrix
>>>>> of a configurable dependency. I didn't know Testcontainer.org has its own
>>>>> Solr module -- https://www.testcontainers.org/modules/solr/ but we
>>>>> implemented that ourselves; not hard.
>>>>> >
>>>>> >>
>>>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>>>> task, and I don’t think plays to open source communities strengths. It’s
>>>>> hard enough to keep all components of Solr up to date with the latest
>>>>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>>>>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>>> >>
>>>>> >> If something is really going to be backcompatible across certain
>>>>> versions, then maybe having it’s own repo makes sense,
>>>>> >
>>>>> >
>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>> atrophy of the module....
>>>>> >
>>>>> >>
>>>>> >> but I suspect it means those components may go stale…. Look at
>>>>> DIH and Velocity components that are moved over to their own repos, both
>>>>> are getting stale, and I’d argue it’s because they don’t live in the main
>>>>> project where all of us have oversight and easy access.
>>>>> >
>>>>> >
>>>>> > Agreed :-(
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> --
>>>> -----------------------------------------------------
>>>> Noble Paul
>>>>
>>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul
>>
>
Re: Separate git repo(s) for Solr modules
Posted by Jan Høydahl <ja...@cominvent.com>.
Folks
1. This is the dev@lucene list, please move discussions over to dev@solr
2. The thread about simultaneous lucene/solr dev is in dev@solr with subject "Proposal to pin the Lucene snapshot version on main"
3. If you want to discuss whether Solr should be a monorepo or multiple repos, start a new thread at dev@solr
Jan
> 4. mai 2021 kl. 20:15 skrev Dawid Weiss <da...@gmail.com>:
>
> A submodule is nothing more than a checkout of a different repository
> (at a given commit). It's not a fork, unless you clone that repository
> and place it somewhere else.
>
> In any case, I agree Solr should stick to official Lucene releases. If
> there is a need to have a "sticky" intermediate tagged release of
> Lucene for internal (development branch) purposes, Cassandra showed
> what seems like an adequate place to host it until an official Lucene
> release is out.
>
> If you're working on both projects at once and wish to modify code in both,
> IDEs in many cases help you out by substituting code/classpaths from multiple
> sources. I know Eclipse can do it, I'm pretty sure you could try to do
> the same with
> IntelliJ.
>
> Dawid
>
> On Tue, May 4, 2021 at 4:40 PM Atri Sharma <at...@apache.org> wrote:
>>
>> I am a bit confused -- if Lucene was to become a sub module of Solr, are we essentially forking Lucene?
>>
>> I am in agreement with Ilan and Houston -- Solr should be depending on Lucene only as a dependency.
>>
>>
>> On Tue, 4 May 2021, 19:52 Noble Paul, <no...@gmail.com> wrote:
>>>
>>> @Ilan Ginzburg
>>>
>>> I don't think the project split is counter productive because we have lucene as a sub module. Solr does not use lucene like a simple library. It's an integral part of Solr
>>>
>>>
>>> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg <il...@gmail.com> wrote:
>>>>
>>>> As with any dependency on any project, you update the dependency project first then consume the updated dependency in Solr.
>>>>
>>>> If the idea is to be able to modify Lucene and Solr in parallel, then the project split is counterproductive.
>>>>
>>>> From the Solr perspective, Lucene and Zookerper are really two “similar”
>>>> dependencies and IMO we should think about them in that way.
>>>>
>>>> Ilan
>>>>
>>>> On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
>>>>>
>>>>> @Houston
>>>>>
>>>>> So, Are you suggesting we should not do that ?
>>>>>
>>>>> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com> wrote:
>>>>>>
>>>>>> In the future we wont be able to “work on both at the same time”, once Lucene 9 is cut. Why not pull that bandaid now?
>>>>>>
>>>>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>>>>>>>
>>>>>>> I'm still struggling to understand the workflow when I'm working on a feature that spans lucene and solr.
>>>>>>>
>>>>>>> I'm yet to see an argument against sub-modules
>>>>>>>
>>>>>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Shoving such a component into lucene-solr repo makes no sense, given its branching strategy is based on master / branch_8x
>>>>>>>>
>>>>>>>> I get how this could cause issues - I def hadn't thought much about
>>>>>>>> multi-version support and branching. But I don't think moving plugins
>>>>>>>> to a separate repo solves that problem for us. If our first class
>>>>>>>> plugins are set up to have different release "lines" that don't line
>>>>>>>> up with major Solr versions, it's only a matter of time before two of
>>>>>>>> those plugins have branch requirements that conflict. Unless I'm
>>>>>>>> missing something here?
>>>>>>>>
>>>>>>>>> I'd prefer that a module only leave our "contribs" area when the concerns/limitations become real. Doing it prematurely could lead to atrophy of the module....
>>>>>>>>
>>>>>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>>>>>> issues that Ishan brought up in his last reply, and they seem like
>>>>>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>>>>>> serious enough that I'd vote to not risk them until this starts to
>>>>>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>>>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>>>>>> versions. IMO that's a much smaller loss.
>>>>>>>>
>>>>>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <ep...@opensourceconnections.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Testing across multiple versions is always very difficult ;-). I recently saw this very interesting approach to using our Dockerized Solr’s to test a component against a number of previous versions of Solr. https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a model for other packages to adopt.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for the link to that Querqy PR. That is *very* similar to what I do at work (minus multi-version testing), and also similar to how I test multiple versions in one of my pet projects by using a CI build matrix of a configurable dependency. I didn't know Testcontainer.org has its own Solr module -- https://www.testcontainers.org/modules/solr/ but we implemented that ourselves; not hard.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Trying to maintain across multiple versions is kind of a Sisyphean task, and I don’t think plays to open source communities strengths. It’s hard enough to keep all components of Solr up to date with the latest Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>>>>>>>>
>>>>>>>>>> If something is really going to be backcompatible across certain versions, then maybe having it’s own repo makes sense,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'd prefer that a module only leave our "contribs" area when the concerns/limitations become real. Doing it prematurely could lead to atrophy of the module....
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> but I suspect it means those components may go stale…. Look at DIH and Velocity components that are moved over to their own repos, both are getting stale, and I’d argue it’s because they don’t live in the main project where all of us have oversight and easy access.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Agreed :-(
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -----------------------------------------------------
>>>>>>> Noble Paul
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -----------------------------------------------------
>>>>> Noble Paul
>>>
>>>
>>>
>>> --
>>> -----------------------------------------------------
>>> Noble Paul
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
Re: Separate git repo(s) for Solr modules
Posted by Dawid Weiss <da...@gmail.com>.
A submodule is nothing more than a checkout of a different repository
(at a given commit). It's not a fork, unless you clone that repository
and place it somewhere else.
In any case, I agree Solr should stick to official Lucene releases. If
there is a need to have a "sticky" intermediate tagged release of
Lucene for internal (development branch) purposes, Cassandra showed
what seems like an adequate place to host it until an official Lucene
release is out.
If you're working on both projects at once and wish to modify code in both,
IDEs in many cases help you out by substituting code/classpaths from multiple
sources. I know Eclipse can do it, I'm pretty sure you could try to do
the same with
IntelliJ.
Dawid
On Tue, May 4, 2021 at 4:40 PM Atri Sharma <at...@apache.org> wrote:
>
> I am a bit confused -- if Lucene was to become a sub module of Solr, are we essentially forking Lucene?
>
> I am in agreement with Ilan and Houston -- Solr should be depending on Lucene only as a dependency.
>
>
> On Tue, 4 May 2021, 19:52 Noble Paul, <no...@gmail.com> wrote:
>>
>> @Ilan Ginzburg
>>
>> I don't think the project split is counter productive because we have lucene as a sub module. Solr does not use lucene like a simple library. It's an integral part of Solr
>>
>>
>> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg <il...@gmail.com> wrote:
>>>
>>> As with any dependency on any project, you update the dependency project first then consume the updated dependency in Solr.
>>>
>>> If the idea is to be able to modify Lucene and Solr in parallel, then the project split is counterproductive.
>>>
>>> From the Solr perspective, Lucene and Zookerper are really two “similar”
>>> dependencies and IMO we should think about them in that way.
>>>
>>> Ilan
>>>
>>> On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
>>>>
>>>> @Houston
>>>>
>>>> So, Are you suggesting we should not do that ?
>>>>
>>>> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com> wrote:
>>>>>
>>>>> In the future we wont be able to “work on both at the same time”, once Lucene 9 is cut. Why not pull that bandaid now?
>>>>>
>>>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>>>>>>
>>>>>> I'm still struggling to understand the workflow when I'm working on a feature that spans lucene and solr.
>>>>>>
>>>>>> I'm yet to see an argument against sub-modules
>>>>>>
>>>>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com> wrote:
>>>>>>>
>>>>>>> > Shoving such a component into lucene-solr repo makes no sense, given its branching strategy is based on master / branch_8x
>>>>>>>
>>>>>>> I get how this could cause issues - I def hadn't thought much about
>>>>>>> multi-version support and branching. But I don't think moving plugins
>>>>>>> to a separate repo solves that problem for us. If our first class
>>>>>>> plugins are set up to have different release "lines" that don't line
>>>>>>> up with major Solr versions, it's only a matter of time before two of
>>>>>>> those plugins have branch requirements that conflict. Unless I'm
>>>>>>> missing something here?
>>>>>>>
>>>>>>> > I'd prefer that a module only leave our "contribs" area when the concerns/limitations become real. Doing it prematurely could lead to atrophy of the module....
>>>>>>>
>>>>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>>>>> issues that Ishan brought up in his last reply, and they seem like
>>>>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>>>>> serious enough that I'd vote to not risk them until this starts to
>>>>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>>>>> versions. IMO that's a much smaller loss.
>>>>>>>
>>>>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org> wrote:
>>>>>>> >
>>>>>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <ep...@opensourceconnections.com> wrote:
>>>>>>> >>
>>>>>>> >> Testing across multiple versions is always very difficult ;-). I recently saw this very interesting approach to using our Dockerized Solr’s to test a component against a number of previous versions of Solr. https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a model for other packages to adopt.
>>>>>>> >
>>>>>>> >
>>>>>>> > Thanks for the link to that Querqy PR. That is *very* similar to what I do at work (minus multi-version testing), and also similar to how I test multiple versions in one of my pet projects by using a CI build matrix of a configurable dependency. I didn't know Testcontainer.org has its own Solr module -- https://www.testcontainers.org/modules/solr/ but we implemented that ourselves; not hard.
>>>>>>> >
>>>>>>> >>
>>>>>>> >> Trying to maintain across multiple versions is kind of a Sisyphean task, and I don’t think plays to open source communities strengths. It’s hard enough to keep all components of Solr up to date with the latest Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>>>>> >>
>>>>>>> >> If something is really going to be backcompatible across certain versions, then maybe having it’s own repo makes sense,
>>>>>>> >
>>>>>>> >
>>>>>>> > I'd prefer that a module only leave our "contribs" area when the concerns/limitations become real. Doing it prematurely could lead to atrophy of the module....
>>>>>>> >
>>>>>>> >>
>>>>>>> >> but I suspect it means those components may go stale…. Look at DIH and Velocity components that are moved over to their own repos, both are getting stale, and I’d argue it’s because they don’t live in the main project where all of us have oversight and easy access.
>>>>>>> >
>>>>>>> >
>>>>>>> > Agreed :-(
>>>>>>> >
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -----------------------------------------------------
>>>>>> Noble Paul
>>>>
>>>>
>>>>
>>>> --
>>>> -----------------------------------------------------
>>>> Noble Paul
>>
>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Separate git repo(s) for Solr modules
Posted by Atri Sharma <at...@apache.org>.
I am a bit confused -- if Lucene was to become a sub module of Solr, are we
essentially forking Lucene?
I am in agreement with Ilan and Houston -- Solr should be depending on
Lucene only as a dependency.
On Tue, 4 May 2021, 19:52 Noble Paul, <no...@gmail.com> wrote:
> @Ilan Ginzburg <il...@gmail.com>
>
> I don't think the project split is counter productive because we have
> lucene as a sub module. Solr does not use lucene like a simple library.
> It's an integral part of Solr
>
>
> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg <il...@gmail.com> wrote:
>
>> As with any dependency on any project, you update the dependency project
>> first then consume the updated dependency in Solr.
>>
>> If the idea is to be able to modify Lucene and Solr in parallel, then the
>> project split is counterproductive.
>>
>> From the Solr perspective, Lucene and Zookerper are really two “similar”
>> dependencies and IMO we should think about them in that way.
>>
>> Ilan
>>
>> On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
>>
>>> @Houston
>>>
>>> So, Are you suggesting we should not do that ?
>>>
>>> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com>
>>> wrote:
>>>
>>>> In the future we wont be able to “work on both at the same time”, once
>>>> Lucene 9 is cut. Why not pull that bandaid now?
>>>>
>>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm still struggling to understand the workflow when I'm working on a
>>>>> feature that spans lucene and solr.
>>>>>
>>>>> I'm yet to see an argument against sub-modules
>>>>>
>>>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> > Shoving such a component into lucene-solr repo makes no sense,
>>>>>> given its branching strategy is based on master / branch_8x
>>>>>>
>>>>>> I get how this could cause issues - I def hadn't thought much about
>>>>>> multi-version support and branching. But I don't think moving plugins
>>>>>> to a separate repo solves that problem for us. If our first class
>>>>>> plugins are set up to have different release "lines" that don't line
>>>>>> up with major Solr versions, it's only a matter of time before two of
>>>>>> those plugins have branch requirements that conflict. Unless I'm
>>>>>> missing something here?
>>>>>>
>>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>>> atrophy of the module....
>>>>>>
>>>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>>>> issues that Ishan brought up in his last reply, and they seem like
>>>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>>>> serious enough that I'd vote to not risk them until this starts to
>>>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>>>> versions. IMO that's a much smaller loss.
>>>>>>
>>>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>>>>> epugh@opensourceconnections.com> wrote:
>>>>>> >>
>>>>>> >> Testing across multiple versions is always very difficult ;-). I
>>>>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>>>>> to test a component against a number of previous versions of Solr.
>>>>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>>>>> model for other packages to adopt.
>>>>>> >
>>>>>> >
>>>>>> > Thanks for the link to that Querqy PR. That is *very* similar to
>>>>>> what I do at work (minus multi-version testing), and also similar to how I
>>>>>> test multiple versions in one of my pet projects by using a CI build matrix
>>>>>> of a configurable dependency. I didn't know Testcontainer.org has its own
>>>>>> Solr module -- https://www.testcontainers.org/modules/solr/ but we
>>>>>> implemented that ourselves; not hard.
>>>>>> >
>>>>>> >>
>>>>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>>>>> task, and I don’t think plays to open source communities strengths. It’s
>>>>>> hard enough to keep all components of Solr up to date with the latest
>>>>>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>>>>>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>>>> >>
>>>>>> >> If something is really going to be backcompatible across certain
>>>>>> versions, then maybe having it’s own repo makes sense,
>>>>>> >
>>>>>> >
>>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>>> atrophy of the module....
>>>>>> >
>>>>>> >>
>>>>>> >> but I suspect it means those components may go stale…. Look at
>>>>>> DIH and Velocity components that are moved over to their own repos, both
>>>>>> are getting stale, and I’d argue it’s because they don’t live in the main
>>>>>> project where all of us have oversight and easy access.
>>>>>> >
>>>>>> >
>>>>>> > Agreed :-(
>>>>>> >
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> -----------------------------------------------------
>>>>> Noble Paul
>>>>>
>>>>
>>>
>>> --
>>> -----------------------------------------------------
>>> Noble Paul
>>>
>>
>
> --
> -----------------------------------------------------
> Noble Paul
>
Re: Separate git repo(s) for Solr modules
Posted by Noble Paul <no...@gmail.com>.
@Ilan Ginzburg <il...@gmail.com>
I don't think the project split is counter productive because we have
lucene as a sub module. Solr does not use lucene like a simple library.
It's an integral part of Solr
On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg <il...@gmail.com> wrote:
> As with any dependency on any project, you update the dependency project
> first then consume the updated dependency in Solr.
>
> If the idea is to be able to modify Lucene and Solr in parallel, then the
> project split is counterproductive.
>
> From the Solr perspective, Lucene and Zookerper are really two “similar”
> dependencies and IMO we should think about them in that way.
>
> Ilan
>
> On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
>
>> @Houston
>>
>> So, Are you suggesting we should not do that ?
>>
>> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com>
>> wrote:
>>
>>> In the future we wont be able to “work on both at the same time”, once
>>> Lucene 9 is cut. Why not pull that bandaid now?
>>>
>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>>>
>>>> I'm still struggling to understand the workflow when I'm working on a
>>>> feature that spans lucene and solr.
>>>>
>>>> I'm yet to see an argument against sub-modules
>>>>
>>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
>>>> wrote:
>>>>
>>>>> > Shoving such a component into lucene-solr repo makes no sense, given
>>>>> its branching strategy is based on master / branch_8x
>>>>>
>>>>> I get how this could cause issues - I def hadn't thought much about
>>>>> multi-version support and branching. But I don't think moving plugins
>>>>> to a separate repo solves that problem for us. If our first class
>>>>> plugins are set up to have different release "lines" that don't line
>>>>> up with major Solr versions, it's only a matter of time before two of
>>>>> those plugins have branch requirements that conflict. Unless I'm
>>>>> missing something here?
>>>>>
>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>> atrophy of the module....
>>>>>
>>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>>> issues that Ishan brought up in his last reply, and they seem like
>>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>>> serious enough that I'd vote to not risk them until this starts to
>>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>>> versions. IMO that's a much smaller loss.
>>>>>
>>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>>>> epugh@opensourceconnections.com> wrote:
>>>>> >>
>>>>> >> Testing across multiple versions is always very difficult ;-). I
>>>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>>>> to test a component against a number of previous versions of Solr.
>>>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>>>> model for other packages to adopt.
>>>>> >
>>>>> >
>>>>> > Thanks for the link to that Querqy PR. That is *very* similar to
>>>>> what I do at work (minus multi-version testing), and also similar to how I
>>>>> test multiple versions in one of my pet projects by using a CI build matrix
>>>>> of a configurable dependency. I didn't know Testcontainer.org has its own
>>>>> Solr module -- https://www.testcontainers.org/modules/solr/ but we
>>>>> implemented that ourselves; not hard.
>>>>> >
>>>>> >>
>>>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>>>> task, and I don’t think plays to open source communities strengths. It’s
>>>>> hard enough to keep all components of Solr up to date with the latest
>>>>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>>>>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>>> >>
>>>>> >> If something is really going to be backcompatible across certain
>>>>> versions, then maybe having it’s own repo makes sense,
>>>>> >
>>>>> >
>>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>>> atrophy of the module....
>>>>> >
>>>>> >>
>>>>> >> but I suspect it means those components may go stale…. Look at
>>>>> DIH and Velocity components that are moved over to their own repos, both
>>>>> are getting stale, and I’d argue it’s because they don’t live in the main
>>>>> project where all of us have oversight and easy access.
>>>>> >
>>>>> >
>>>>> > Agreed :-(
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> --
>>>> -----------------------------------------------------
>>>> Noble Paul
>>>>
>>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul
>>
>
--
-----------------------------------------------------
Noble Paul
Re: Separate git repo(s) for Solr modules
Posted by Ilan Ginzburg <il...@gmail.com>.
As with any dependency on any project, you update the dependency project
first then consume the updated dependency in Solr.
If the idea is to be able to modify Lucene and Solr in parallel, then the
project split is counterproductive.
From the Solr perspective, Lucene and Zookerper are really two “similar”
dependencies and IMO we should think about them in that way.
Ilan
On Tue 4 May 2021 at 09:45, Noble Paul <no...@gmail.com> wrote:
> @Houston
>
> So, Are you suggesting we should not do that ?
>
> On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com>
> wrote:
>
>> In the future we wont be able to “work on both at the same time”, once
>> Lucene 9 is cut. Why not pull that bandaid now?
>>
>> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>>
>>> I'm still struggling to understand the workflow when I'm working on a
>>> feature that spans lucene and solr.
>>>
>>> I'm yet to see an argument against sub-modules
>>>
>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
>>> wrote:
>>>
>>>> > Shoving such a component into lucene-solr repo makes no sense, given
>>>> its branching strategy is based on master / branch_8x
>>>>
>>>> I get how this could cause issues - I def hadn't thought much about
>>>> multi-version support and branching. But I don't think moving plugins
>>>> to a separate repo solves that problem for us. If our first class
>>>> plugins are set up to have different release "lines" that don't line
>>>> up with major Solr versions, it's only a matter of time before two of
>>>> those plugins have branch requirements that conflict. Unless I'm
>>>> missing something here?
>>>>
>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>> atrophy of the module....
>>>>
>>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>>> issues that Ishan brought up in his last reply, and they seem like
>>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>>> serious enough that I'd vote to not risk them until this starts to
>>>> cause us issues in practice. Even if, for now, that means we won't be
>>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>>> versions. IMO that's a much smaller loss.
>>>>
>>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org>
>>>> wrote:
>>>> >
>>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>>> epugh@opensourceconnections.com> wrote:
>>>> >>
>>>> >> Testing across multiple versions is always very difficult ;-). I
>>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>>> to test a component against a number of previous versions of Solr.
>>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>>> model for other packages to adopt.
>>>> >
>>>> >
>>>> > Thanks for the link to that Querqy PR. That is *very* similar to
>>>> what I do at work (minus multi-version testing), and also similar to how I
>>>> test multiple versions in one of my pet projects by using a CI build matrix
>>>> of a configurable dependency. I didn't know Testcontainer.org has its own
>>>> Solr module -- https://www.testcontainers.org/modules/solr/ but we
>>>> implemented that ourselves; not hard.
>>>> >
>>>> >>
>>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>>> task, and I don’t think plays to open source communities strengths. It’s
>>>> hard enough to keep all components of Solr up to date with the latest
>>>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>>>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>>>> >>
>>>> >> If something is really going to be backcompatible across certain
>>>> versions, then maybe having it’s own repo makes sense,
>>>> >
>>>> >
>>>> > I'd prefer that a module only leave our "contribs" area when the
>>>> concerns/limitations become real. Doing it prematurely could lead to
>>>> atrophy of the module....
>>>> >
>>>> >>
>>>> >> but I suspect it means those components may go stale…. Look at DIH
>>>> and Velocity components that are moved over to their own repos, both are
>>>> getting stale, and I’d argue it’s because they don’t live in the main
>>>> project where all of us have oversight and easy access.
>>>> >
>>>> >
>>>> > Agreed :-(
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>> --
>>> -----------------------------------------------------
>>> Noble Paul
>>>
>>
>
> --
> -----------------------------------------------------
> Noble Paul
>
Re: Separate git repo(s) for Solr modules
Posted by Noble Paul <no...@gmail.com>.
@Houston
So, Are you suggesting we should not do that ?
On Tue, May 4, 2021 at 2:35 PM Houston Putman <ho...@gmail.com>
wrote:
> In the future we wont be able to “work on both at the same time”, once
> Lucene 9 is cut. Why not pull that bandaid now?
>
> On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
>
>> I'm still struggling to understand the workflow when I'm working on a
>> feature that spans lucene and solr.
>>
>> I'm yet to see an argument against sub-modules
>>
>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
>> wrote:
>>
>>> > Shoving such a component into lucene-solr repo makes no sense, given
>>> its branching strategy is based on master / branch_8x
>>>
>>> I get how this could cause issues - I def hadn't thought much about
>>> multi-version support and branching. But I don't think moving plugins
>>> to a separate repo solves that problem for us. If our first class
>>> plugins are set up to have different release "lines" that don't line
>>> up with major Solr versions, it's only a matter of time before two of
>>> those plugins have branch requirements that conflict. Unless I'm
>>> missing something here?
>>>
>>> > I'd prefer that a module only leave our "contribs" area when the
>>> concerns/limitations become real. Doing it prematurely could lead to
>>> atrophy of the module....
>>>
>>> +1 to David's comment. I def hadn't considered the branching-scheme
>>> issues that Ishan brought up in his last reply, and they seem like
>>> valid concerns to me. But the risk and the downsides of "atrophy" are
>>> serious enough that I'd vote to not risk them until this starts to
>>> cause us issues in practice. Even if, for now, that means we won't be
>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>> versions. IMO that's a much smaller loss.
>>>
>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org> wrote:
>>> >
>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>> epugh@opensourceconnections.com> wrote:
>>> >>
>>> >> Testing across multiple versions is always very difficult ;-). I
>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>> to test a component against a number of previous versions of Solr.
>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>> model for other packages to adopt.
>>> >
>>> >
>>> > Thanks for the link to that Querqy PR. That is *very* similar to what
>>> I do at work (minus multi-version testing), and also similar to how I test
>>> multiple versions in one of my pet projects by using a CI build matrix of a
>>> configurable dependency. I didn't know Testcontainer.org has its own Solr
>>> module -- https://www.testcontainers.org/modules/solr/ but we
>>> implemented that ourselves; not hard.
>>> >
>>> >>
>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>> task, and I don’t think plays to open source communities strengths. It’s
>>> hard enough to keep all components of Solr up to date with the latest
>>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>>> >>
>>> >> If something is really going to be backcompatible across certain
>>> versions, then maybe having it’s own repo makes sense,
>>> >
>>> >
>>> > I'd prefer that a module only leave our "contribs" area when the
>>> concerns/limitations become real. Doing it prematurely could lead to
>>> atrophy of the module....
>>> >
>>> >>
>>> >> but I suspect it means those components may go stale…. Look at DIH
>>> and Velocity components that are moved over to their own repos, both are
>>> getting stale, and I’d argue it’s because they don’t live in the main
>>> project where all of us have oversight and easy access.
>>> >
>>> >
>>> > Agreed :-(
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul
>>
>
--
-----------------------------------------------------
Noble Paul
Re: Separate git repo(s) for Solr modules
Posted by Houston Putman <ho...@gmail.com>.
In the future we wont be able to “work on both at the same time”, once
Lucene 9 is cut. Why not pull that bandaid now?
On Mon, May 3, 2021 at 11:32 PM Noble Paul <no...@gmail.com> wrote:
> I'm still struggling to understand the workflow when I'm working on a
> feature that spans lucene and solr.
>
> I'm yet to see an argument against sub-modules
>
> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
>
>> > Shoving such a component into lucene-solr repo makes no sense, given
>> its branching strategy is based on master / branch_8x
>>
>> I get how this could cause issues - I def hadn't thought much about
>> multi-version support and branching. But I don't think moving plugins
>> to a separate repo solves that problem for us. If our first class
>> plugins are set up to have different release "lines" that don't line
>> up with major Solr versions, it's only a matter of time before two of
>> those plugins have branch requirements that conflict. Unless I'm
>> missing something here?
>>
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real. Doing it prematurely could lead to
>> atrophy of the module....
>>
>> +1 to David's comment. I def hadn't considered the branching-scheme
>> issues that Ishan brought up in his last reply, and they seem like
>> valid concerns to me. But the risk and the downsides of "atrophy" are
>> serious enough that I'd vote to not risk them until this starts to
>> cause us issues in practice. Even if, for now, that means we won't be
>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>> versions. IMO that's a much smaller loss.
>>
>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley <ds...@apache.org> wrote:
>> >
>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>> epugh@opensourceconnections.com> wrote:
>> >>
>> >> Testing across multiple versions is always very difficult ;-). I
>> recently saw this very interesting approach to using our Dockerized Solr’s
>> to test a component against a number of previous versions of Solr.
>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>> model for other packages to adopt.
>> >
>> >
>> > Thanks for the link to that Querqy PR. That is *very* similar to what
>> I do at work (minus multi-version testing), and also similar to how I test
>> multiple versions in one of my pet projects by using a CI build matrix of a
>> configurable dependency. I didn't know Testcontainer.org has its own Solr
>> module -- https://www.testcontainers.org/modules/solr/ but we
>> implemented that ourselves; not hard.
>> >
>> >>
>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>> task, and I don’t think plays to open source communities strengths. It’s
>> hard enough to keep all components of Solr up to date with the latest
>> Lucene and the latest Solr…. (Try out wt=xlsx Response Writer, it’s
>> currently broken on master) . Reminds me of the Apache Gump project ;-)
>> >>
>> >> If something is really going to be backcompatible across certain
>> versions, then maybe having it’s own repo makes sense,
>> >
>> >
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real. Doing it prematurely could lead to
>> atrophy of the module....
>> >
>> >>
>> >> but I suspect it means those components may go stale…. Look at DIH
>> and Velocity components that are moved over to their own repos, both are
>> getting stale, and I’d argue it’s because they don’t live in the main
>> project where all of us have oversight and easy access.
>> >
>> >
>> > Agreed :-(
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
> --
> -----------------------------------------------------
> Noble Paul
>