You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by Christos Malliaridis <ma...@apache.org> on 2025/03/26 17:04:13 UTC

Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Hello everyone,

In consultation with David Smiley, I am planning to request an OS upgrade
for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
31, as they are running older operating systems and causing OS-related
build errors in the Solr project.

Since the nodes will be replaced with new instances, they will need to be
reconfigured for both projects. Please speak up if this may be an issue or
if I should wait longer before requesting an upgrade from infra. I will
also need someone with access to each project to configure the nodes again,
as I do not currently have sufficient access.

Best,
Christos

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
I've added a comment on [1] asking how we should handle snapshot uploads,
which are currently failing.

[1] https://issues.apache.org/jira/browse/INFRA-26732

On Tue, Apr 22, 2025 at 8:04 PM Dawid Weiss <da...@gmail.com> wrote:

> Nice! I can't think of anything. I guess we just let it burn? I can't
> see any downside to just letting it go - it's just one more CI.
>
> D.
>
> On Tue, Apr 22, 2025 at 7:30 PM Christos Malliaridis
> <c....@gmail.com> wrote:
> >
> > Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 –
> to replace the old ones. They come with upgraded hardware:
> >
> >> 16GB -> 32GB RAM
> >> 4CPU -> 16CPU
> >> 256/350GB -> 500GB disk
> >
> >
> > Once we are ready to turn off the old nodes, I'll respond to the Jira
> issue.
> >
> > What steps are still remaining before we can turn off the old nodes?
> >
> > Best,
> > Christos
> >
> > On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <
> c.malliaridis@gmail.com> wrote:
> >>
> >> I've requested an upgrade here:
> https://issues.apache.org/jira/browse/INFRA-26732
> >>
> >> I hope I didn't forget anything. :)
> >>
> >> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
> >>>
> >>> Can you make sure that the same people get sudo/root acces slike
> before?
> >>> It would also be ideal to maybe ask them to preserve the jenkins
> >>> directory. I cleaned them up a bit already.
> >>>
> >>> If we get a new machine, could you ask them for more power and memory?
> >>> The current machines are very slow, looks like overprovisioned VMware!
> >>>
> >>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> >>> > Hi,
> >>> >
> >>> > go ahead. I preserved some config files. Actually the Gradle builds
> >>> > are easier to maintain as Ant, so basically it is just 2 files.
> >>> >
> >>> > Uwe
> >>> >
> >>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> >>> >> Oh sorry, will do! I ping you.
> >>> >>
> >>> >> Uwe
> >>> >>
> >>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
> >>> >>> @Uwe Did you have the time to run a backup of the home directory? I
> >>> >>> would
> >>> >>> proceed and open an infra ticket if we are ready, I just need to
> >>> >>> know what "all
> >>> >>> packages we
> >>> >>> need for Jenkins Slaves" includes, as I am not sure how to look
> this
> >>> >>> up to
> >>> >>> include it in the ticket.
> >>> >>>
> >>> >>> Christos
> >>> >>>
> >>> >>> Hi,
> >>> >>>
> >>> >>>
> >>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
> >>> >>>> Unfortunately
> >>> >>>> it does not work, as there are too many PPA repositories in the
> config
> >>> >>>> (nodejs,....) and those lead to package conflicts so
> >>> >>>> do-release-upgrade
> >>> >>>> bails out.
> >>> >>>
> >>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't
> know
> >>> >>>> how
> >>> >>>> to get puppet to reinstallthem after upgrade.
> >>> >>>
> >>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
> >>> >>>> provided
> >>> >>>> by ASF. That's the problem of nailed down instances with horrible
> >>> >>>> puppet
> >>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all
> of my
> >>> >>>> customers, it always causes issues administering machines for Solr
> >>> >>>> installations!).
> >>> >>>
> >>> >>>> I can do the jenkins home directory backup tomorrow and then we
> can
> >>> >>>> ask
> >>> >>> them to reinstall newer OS. We just need to make sure to tell them
> that
> >>> >>>> they should use a puppet configuration that contains all packages
> we
> >>> >>> need for Jenkins Slaves.
> >>> >>>
> >>> >>>
> >>> >>>> Uwe
> >>> >>
> >>> --
> >>> Uwe Schindler
> >>> Achterdiek 19, D-28357 Bremen
> >>> https://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
I've added a comment on [1] asking how we should handle snapshot uploads,
which are currently failing.

[1] https://issues.apache.org/jira/browse/INFRA-26732

On Tue, Apr 22, 2025 at 8:04 PM Dawid Weiss <da...@gmail.com> wrote:

> Nice! I can't think of anything. I guess we just let it burn? I can't
> see any downside to just letting it go - it's just one more CI.
>
> D.
>
> On Tue, Apr 22, 2025 at 7:30 PM Christos Malliaridis
> <c....@gmail.com> wrote:
> >
> > Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 –
> to replace the old ones. They come with upgraded hardware:
> >
> >> 16GB -> 32GB RAM
> >> 4CPU -> 16CPU
> >> 256/350GB -> 500GB disk
> >
> >
> > Once we are ready to turn off the old nodes, I'll respond to the Jira
> issue.
> >
> > What steps are still remaining before we can turn off the old nodes?
> >
> > Best,
> > Christos
> >
> > On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <
> c.malliaridis@gmail.com> wrote:
> >>
> >> I've requested an upgrade here:
> https://issues.apache.org/jira/browse/INFRA-26732
> >>
> >> I hope I didn't forget anything. :)
> >>
> >> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
> >>>
> >>> Can you make sure that the same people get sudo/root acces slike
> before?
> >>> It would also be ideal to maybe ask them to preserve the jenkins
> >>> directory. I cleaned them up a bit already.
> >>>
> >>> If we get a new machine, could you ask them for more power and memory?
> >>> The current machines are very slow, looks like overprovisioned VMware!
> >>>
> >>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> >>> > Hi,
> >>> >
> >>> > go ahead. I preserved some config files. Actually the Gradle builds
> >>> > are easier to maintain as Ant, so basically it is just 2 files.
> >>> >
> >>> > Uwe
> >>> >
> >>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> >>> >> Oh sorry, will do! I ping you.
> >>> >>
> >>> >> Uwe
> >>> >>
> >>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
> >>> >>> @Uwe Did you have the time to run a backup of the home directory? I
> >>> >>> would
> >>> >>> proceed and open an infra ticket if we are ready, I just need to
> >>> >>> know what "all
> >>> >>> packages we
> >>> >>> need for Jenkins Slaves" includes, as I am not sure how to look
> this
> >>> >>> up to
> >>> >>> include it in the ticket.
> >>> >>>
> >>> >>> Christos
> >>> >>>
> >>> >>> Hi,
> >>> >>>
> >>> >>>
> >>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
> >>> >>>> Unfortunately
> >>> >>>> it does not work, as there are too many PPA repositories in the
> config
> >>> >>>> (nodejs,....) and those lead to package conflicts so
> >>> >>>> do-release-upgrade
> >>> >>>> bails out.
> >>> >>>
> >>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't
> know
> >>> >>>> how
> >>> >>>> to get puppet to reinstallthem after upgrade.
> >>> >>>
> >>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
> >>> >>>> provided
> >>> >>>> by ASF. That's the problem of nailed down instances with horrible
> >>> >>>> puppet
> >>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all
> of my
> >>> >>>> customers, it always causes issues administering machines for Solr
> >>> >>>> installations!).
> >>> >>>
> >>> >>>> I can do the jenkins home directory backup tomorrow and then we
> can
> >>> >>>> ask
> >>> >>> them to reinstall newer OS. We just need to make sure to tell them
> that
> >>> >>>> they should use a puppet configuration that contains all packages
> we
> >>> >>> need for Jenkins Slaves.
> >>> >>>
> >>> >>>
> >>> >>>> Uwe
> >>> >>
> >>> --
> >>> Uwe Schindler
> >>> Achterdiek 19, D-28357 Bremen
> >>> https://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
Nice! I can't think of anything. I guess we just let it burn? I can't
see any downside to just letting it go - it's just one more CI.

D.

On Tue, Apr 22, 2025 at 7:30 PM Christos Malliaridis
<c....@gmail.com> wrote:
>
> Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 – to replace the old ones. They come with upgraded hardware:
>
>> 16GB -> 32GB RAM
>> 4CPU -> 16CPU
>> 256/350GB -> 500GB disk
>
>
> Once we are ready to turn off the old nodes, I'll respond to the Jira issue.
>
> What steps are still remaining before we can turn off the old nodes?
>
> Best,
> Christos
>
> On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <c....@gmail.com> wrote:
>>
>> I've requested an upgrade here: https://issues.apache.org/jira/browse/INFRA-26732
>>
>> I hope I didn't forget anything. :)
>>
>> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
>>>
>>> Can you make sure that the same people get sudo/root acces slike before?
>>> It would also be ideal to maybe ask them to preserve the jenkins
>>> directory. I cleaned them up a bit already.
>>>
>>> If we get a new machine, could you ask them for more power and memory?
>>> The current machines are very slow, looks like overprovisioned VMware!
>>>
>>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
>>> > Hi,
>>> >
>>> > go ahead. I preserved some config files. Actually the Gradle builds
>>> > are easier to maintain as Ant, so basically it is just 2 files.
>>> >
>>> > Uwe
>>> >
>>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>>> >> Oh sorry, will do! I ping you.
>>> >>
>>> >> Uwe
>>> >>
>>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>>> >>> @Uwe Did you have the time to run a backup of the home directory? I
>>> >>> would
>>> >>> proceed and open an infra ticket if we are ready, I just need to
>>> >>> know what "all
>>> >>> packages we
>>> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
>>> >>> up to
>>> >>> include it in the ticket.
>>> >>>
>>> >>> Christos
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>>
>>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
>>> >>>> Unfortunately
>>> >>>> it does not work, as there are too many PPA repositories in the config
>>> >>>> (nodejs,....) and those lead to package conflicts so
>>> >>>> do-release-upgrade
>>> >>>> bails out.
>>> >>>
>>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
>>> >>>> how
>>> >>>> to get puppet to reinstallthem after upgrade.
>>> >>>
>>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
>>> >>>> provided
>>> >>>> by ASF. That's the problem of nailed down instances with horrible
>>> >>>> puppet
>>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>> >>>> customers, it always causes issues administering machines for Solr
>>> >>>> installations!).
>>> >>>
>>> >>>> I can do the jenkins home directory backup tomorrow and then we can
>>> >>>> ask
>>> >>> them to reinstall newer OS. We just need to make sure to tell them that
>>> >>>> they should use a puppet configuration that contains all packages we
>>> >>> need for Jenkins Slaves.
>>> >>>
>>> >>>
>>> >>>> Uwe
>>> >>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, D-28357 Bremen
>>> https://www.thetaphi.de
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
Nice! I can't think of anything. I guess we just let it burn? I can't
see any downside to just letting it go - it's just one more CI.

D.

On Tue, Apr 22, 2025 at 7:30 PM Christos Malliaridis
<c....@gmail.com> wrote:
>
> Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 – to replace the old ones. They come with upgraded hardware:
>
>> 16GB -> 32GB RAM
>> 4CPU -> 16CPU
>> 256/350GB -> 500GB disk
>
>
> Once we are ready to turn off the old nodes, I'll respond to the Jira issue.
>
> What steps are still remaining before we can turn off the old nodes?
>
> Best,
> Christos
>
> On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <c....@gmail.com> wrote:
>>
>> I've requested an upgrade here: https://issues.apache.org/jira/browse/INFRA-26732
>>
>> I hope I didn't forget anything. :)
>>
>> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
>>>
>>> Can you make sure that the same people get sudo/root acces slike before?
>>> It would also be ideal to maybe ask them to preserve the jenkins
>>> directory. I cleaned them up a bit already.
>>>
>>> If we get a new machine, could you ask them for more power and memory?
>>> The current machines are very slow, looks like overprovisioned VMware!
>>>
>>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
>>> > Hi,
>>> >
>>> > go ahead. I preserved some config files. Actually the Gradle builds
>>> > are easier to maintain as Ant, so basically it is just 2 files.
>>> >
>>> > Uwe
>>> >
>>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>>> >> Oh sorry, will do! I ping you.
>>> >>
>>> >> Uwe
>>> >>
>>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>>> >>> @Uwe Did you have the time to run a backup of the home directory? I
>>> >>> would
>>> >>> proceed and open an infra ticket if we are ready, I just need to
>>> >>> know what "all
>>> >>> packages we
>>> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
>>> >>> up to
>>> >>> include it in the ticket.
>>> >>>
>>> >>> Christos
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>>
>>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
>>> >>>> Unfortunately
>>> >>>> it does not work, as there are too many PPA repositories in the config
>>> >>>> (nodejs,....) and those lead to package conflicts so
>>> >>>> do-release-upgrade
>>> >>>> bails out.
>>> >>>
>>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
>>> >>>> how
>>> >>>> to get puppet to reinstallthem after upgrade.
>>> >>>
>>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
>>> >>>> provided
>>> >>>> by ASF. That's the problem of nailed down instances with horrible
>>> >>>> puppet
>>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>> >>>> customers, it always causes issues administering machines for Solr
>>> >>>> installations!).
>>> >>>
>>> >>>> I can do the jenkins home directory backup tomorrow and then we can
>>> >>>> ask
>>> >>> them to reinstall newer OS. We just need to make sure to tell them that
>>> >>>> they should use a puppet configuration that contains all packages we
>>> >>> need for Jenkins Slaves.
>>> >>>
>>> >>>
>>> >>>> Uwe
>>> >>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, D-28357 Bremen
>>> https://www.thetaphi.de
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 – to
replace the old ones. They come with upgraded hardware:

16GB -> 32GB RAM
> 4CPU -> 16CPU
> 256/350GB -> 500GB disk


Once we are ready to turn off the old nodes, I'll respond to the Jira issue.

*What steps are still remaining before we can turn off the old nodes?*

Best,
Christos

On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <c....@gmail.com>
wrote:

> I've requested an upgrade here:
> https://issues.apache.org/jira/browse/INFRA-26732
>
> I hope I didn't forget anything. :)
>
> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
>
>> Can you make sure that the same people get sudo/root acces slike before?
>> It would also be ideal to maybe ask them to preserve the jenkins
>> directory. I cleaned them up a bit already.
>>
>> If we get a new machine, could you ask them for more power and memory?
>> The current machines are very slow, looks like overprovisioned VMware!
>>
>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
>> > Hi,
>> >
>> > go ahead. I preserved some config files. Actually the Gradle builds
>> > are easier to maintain as Ant, so basically it is just 2 files.
>> >
>> > Uwe
>> >
>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>> >> Oh sorry, will do! I ping you.
>> >>
>> >> Uwe
>> >>
>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>> >>> @Uwe Did you have the time to run a backup of the home directory? I
>> >>> would
>> >>> proceed and open an infra ticket if we are ready, I just need to
>> >>> know what "all
>> >>> packages we
>> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
>> >>> up to
>> >>> include it in the ticket.
>> >>>
>> >>> Christos
>> >>>
>> >>> Hi,
>> >>>
>> >>>
>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
>> >>>> Unfortunately
>> >>>> it does not work, as there are too many PPA repositories in the
>> config
>> >>>> (nodejs,....) and those lead to package conflicts so
>> >>>> do-release-upgrade
>> >>>> bails out.
>> >>>
>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
>> >>>> how
>> >>>> to get puppet to reinstallthem after upgrade.
>> >>>
>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
>> >>>> provided
>> >>>> by ASF. That's the problem of nailed down instances with horrible
>> >>>> puppet
>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>> >>>> customers, it always causes issues administering machines for Solr
>> >>>> installations!).
>> >>>
>> >>>> I can do the jenkins home directory backup tomorrow and then we can
>> >>>> ask
>> >>> them to reinstall newer OS. We just need to make sure to tell them
>> that
>> >>>> they should use a puppet configuration that contains all packages we
>> >>> need for Jenkins Slaves.
>> >>>
>> >>>
>> >>>> Uwe
>> >>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
Infra provided us with two new nodes – lucene-solr-3 and lucene-solr-4 – to
replace the old ones. They come with upgraded hardware:

16GB -> 32GB RAM
> 4CPU -> 16CPU
> 256/350GB -> 500GB disk


Once we are ready to turn off the old nodes, I'll respond to the Jira issue.

*What steps are still remaining before we can turn off the old nodes?*

Best,
Christos

On Mon, Apr 7, 2025 at 8:09 PM Christos Malliaridis <c....@gmail.com>
wrote:

> I've requested an upgrade here:
> https://issues.apache.org/jira/browse/INFRA-26732
>
> I hope I didn't forget anything. :)
>
> On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:
>
>> Can you make sure that the same people get sudo/root acces slike before?
>> It would also be ideal to maybe ask them to preserve the jenkins
>> directory. I cleaned them up a bit already.
>>
>> If we get a new machine, could you ask them for more power and memory?
>> The current machines are very slow, looks like overprovisioned VMware!
>>
>> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
>> > Hi,
>> >
>> > go ahead. I preserved some config files. Actually the Gradle builds
>> > are easier to maintain as Ant, so basically it is just 2 files.
>> >
>> > Uwe
>> >
>> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>> >> Oh sorry, will do! I ping you.
>> >>
>> >> Uwe
>> >>
>> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>> >>> @Uwe Did you have the time to run a backup of the home directory? I
>> >>> would
>> >>> proceed and open an infra ticket if we are ready, I just need to
>> >>> know what "all
>> >>> packages we
>> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
>> >>> up to
>> >>> include it in the ticket.
>> >>>
>> >>> Christos
>> >>>
>> >>> Hi,
>> >>>
>> >>>
>> >>>> I tried the do-release-upgrade on the older 18.04 machine.
>> >>>> Unfortunately
>> >>>> it does not work, as there are too many PPA repositories in the
>> config
>> >>>> (nodejs,....) and those lead to package conflicts so
>> >>>> do-release-upgrade
>> >>>> bails out.
>> >>>
>> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
>> >>>> how
>> >>>> to get puppet to reinstallthem after upgrade.
>> >>>
>> >>>> So it looks like we need to do the "rebuild node" with fresh OS
>> >>>> provided
>> >>>> by ASF. That's the problem of nailed down instances with horrible
>> >>>> puppet
>> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>> >>>> customers, it always causes issues administering machines for Solr
>> >>>> installations!).
>> >>>
>> >>>> I can do the jenkins home directory backup tomorrow and then we can
>> >>>> ask
>> >>> them to reinstall newer OS. We just need to make sure to tell them
>> that
>> >>>> they should use a puppet configuration that contains all packages we
>> >>> need for Jenkins Slaves.
>> >>>
>> >>>
>> >>>> Uwe
>> >>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
I've requested an upgrade here:
https://issues.apache.org/jira/browse/INFRA-26732

I hope I didn't forget anything. :)

On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:

> Can you make sure that the same people get sudo/root acces slike before?
> It would also be ideal to maybe ask them to preserve the jenkins
> directory. I cleaned them up a bit already.
>
> If we get a new machine, could you ask them for more power and memory?
> The current machines are very slow, looks like overprovisioned VMware!
>
> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> > Hi,
> >
> > go ahead. I preserved some config files. Actually the Gradle builds
> > are easier to maintain as Ant, so basically it is just 2 files.
> >
> > Uwe
> >
> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> >> Oh sorry, will do! I ping you.
> >>
> >> Uwe
> >>
> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
> >>> @Uwe Did you have the time to run a backup of the home directory? I
> >>> would
> >>> proceed and open an infra ticket if we are ready, I just need to
> >>> know what "all
> >>> packages we
> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
> >>> up to
> >>> include it in the ticket.
> >>>
> >>> Christos
> >>>
> >>> Hi,
> >>>
> >>>
> >>>> I tried the do-release-upgrade on the older 18.04 machine.
> >>>> Unfortunately
> >>>> it does not work, as there are too many PPA repositories in the config
> >>>> (nodejs,....) and those lead to package conflicts so
> >>>> do-release-upgrade
> >>>> bails out.
> >>>
> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
> >>>> how
> >>>> to get puppet to reinstallthem after upgrade.
> >>>
> >>>> So it looks like we need to do the "rebuild node" with fresh OS
> >>>> provided
> >>>> by ASF. That's the problem of nailed down instances with horrible
> >>>> puppet
> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
> >>>> customers, it always causes issues administering machines for Solr
> >>>> installations!).
> >>>
> >>>> I can do the jenkins home directory backup tomorrow and then we can
> >>>> ask
> >>> them to reinstall newer OS. We just need to make sure to tell them that
> >>>> they should use a puppet configuration that contains all packages we
> >>> need for Jenkins Slaves.
> >>>
> >>>
> >>>> Uwe
> >>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
I've requested an upgrade here:
https://issues.apache.org/jira/browse/INFRA-26732

I hope I didn't forget anything. :)

On Wed, Apr 2, 2025 at 10:19 PM Uwe Schindler <uw...@thetaphi.de> wrote:

> Can you make sure that the same people get sudo/root acces slike before?
> It would also be ideal to maybe ask them to preserve the jenkins
> directory. I cleaned them up a bit already.
>
> If we get a new machine, could you ask them for more power and memory?
> The current machines are very slow, looks like overprovisioned VMware!
>
> Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> > Hi,
> >
> > go ahead. I preserved some config files. Actually the Gradle builds
> > are easier to maintain as Ant, so basically it is just 2 files.
> >
> > Uwe
> >
> > Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> >> Oh sorry, will do! I ping you.
> >>
> >> Uwe
> >>
> >> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
> >>> @Uwe Did you have the time to run a backup of the home directory? I
> >>> would
> >>> proceed and open an infra ticket if we are ready, I just need to
> >>> know what "all
> >>> packages we
> >>> need for Jenkins Slaves" includes, as I am not sure how to look this
> >>> up to
> >>> include it in the ticket.
> >>>
> >>> Christos
> >>>
> >>> Hi,
> >>>
> >>>
> >>>> I tried the do-release-upgrade on the older 18.04 machine.
> >>>> Unfortunately
> >>>> it does not work, as there are too many PPA repositories in the config
> >>>> (nodejs,....) and those lead to package conflicts so
> >>>> do-release-upgrade
> >>>> bails out.
> >>>
> >>>> I could uninstall all those PPAs with "ppa-purge", but I don't know
> >>>> how
> >>>> to get puppet to reinstallthem after upgrade.
> >>>
> >>>> So it looks like we need to do the "rebuild node" with fresh OS
> >>>> provided
> >>>> by ASF. That's the problem of nailed down instances with horrible
> >>>> puppet
> >>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
> >>>> customers, it always causes issues administering machines for Solr
> >>>> installations!).
> >>>
> >>>> I can do the jenkins home directory backup tomorrow and then we can
> >>>> ask
> >>> them to reinstall newer OS. We just need to make sure to tell them that
> >>>> they should use a puppet configuration that contains all packages we
> >>> need for Jenkins Slaves.
> >>>
> >>>
> >>>> Uwe
> >>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Can you make sure that the same people get sudo/root acces slike before? 
It would also be ideal to maybe ask them to preserve the jenkins 
directory. I cleaned them up a bit already.

If we get a new machine, could you ask them for more power and memory? 
The current machines are very slow, looks like overprovisioned VMware!

Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> Hi,
>
> go ahead. I preserved some config files. Actually the Gradle builds 
> are easier to maintain as Ant, so basically it is just 2 files.
>
> Uwe
>
> Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>> Oh sorry, will do! I ping you.
>>
>> Uwe
>>
>> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>>> @Uwe Did you have the time to run a backup of the home directory? I 
>>> would
>>> proceed and open an infra ticket if we are ready, I just need to 
>>> know what "all
>>> packages we
>>> need for Jenkins Slaves" includes, as I am not sure how to look this 
>>> up to
>>> include it in the ticket.
>>>
>>> Christos
>>>
>>> Hi,
>>>
>>>
>>>> I tried the do-release-upgrade on the older 18.04 machine. 
>>>> Unfortunately
>>>> it does not work, as there are too many PPA repositories in the config
>>>> (nodejs,....) and those lead to package conflicts so 
>>>> do-release-upgrade
>>>> bails out.
>>>
>>>> I could uninstall all those PPAs with "ppa-purge", but I don't know 
>>>> how
>>>> to get puppet to reinstallthem after upgrade.
>>>
>>>> So it looks like we need to do the "rebuild node" with fresh OS 
>>>> provided
>>>> by ASF. That's the problem of nailed down instances with horrible 
>>>> puppet
>>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>>> customers, it always causes issues administering machines for Solr
>>>> installations!).
>>>
>>>> I can do the jenkins home directory backup tomorrow and then we can 
>>>> ask
>>> them to reinstall newer OS. We just need to make sure to tell them that
>>>> they should use a puppet configuration that contains all packages we
>>> need for Jenkins Slaves.
>>>
>>>
>>>> Uwe
>>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Can you make sure that the same people get sudo/root acces slike before? 
It would also be ideal to maybe ask them to preserve the jenkins 
directory. I cleaned them up a bit already.

If we get a new machine, could you ask them for more power and memory? 
The current machines are very slow, looks like overprovisioned VMware!

Am 02.04.2025 um 21:17 schrieb Uwe Schindler:
> Hi,
>
> go ahead. I preserved some config files. Actually the Gradle builds 
> are easier to maintain as Ant, so basically it is just 2 files.
>
> Uwe
>
> Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
>> Oh sorry, will do! I ping you.
>>
>> Uwe
>>
>> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>>> @Uwe Did you have the time to run a backup of the home directory? I 
>>> would
>>> proceed and open an infra ticket if we are ready, I just need to 
>>> know what "all
>>> packages we
>>> need for Jenkins Slaves" includes, as I am not sure how to look this 
>>> up to
>>> include it in the ticket.
>>>
>>> Christos
>>>
>>> Hi,
>>>
>>>
>>>> I tried the do-release-upgrade on the older 18.04 machine. 
>>>> Unfortunately
>>>> it does not work, as there are too many PPA repositories in the config
>>>> (nodejs,....) and those lead to package conflicts so 
>>>> do-release-upgrade
>>>> bails out.
>>>
>>>> I could uninstall all those PPAs with "ppa-purge", but I don't know 
>>>> how
>>>> to get puppet to reinstallthem after upgrade.
>>>
>>>> So it looks like we need to do the "rebuild node" with fresh OS 
>>>> provided
>>>> by ASF. That's the problem of nailed down instances with horrible 
>>>> puppet
>>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>>> customers, it always causes issues administering machines for Solr
>>>> installations!).
>>>
>>>> I can do the jenkins home directory backup tomorrow and then we can 
>>>> ask
>>> them to reinstall newer OS. We just need to make sure to tell them that
>>>> they should use a puppet configuration that contains all packages we
>>> need for Jenkins Slaves.
>>>
>>>
>>>> Uwe
>>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

go ahead. I preserved some config files. Actually the Gradle builds are 
easier to maintain as Ant, so basically it is just 2 files.

Uwe

Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> Oh sorry, will do! I ping you.
>
> Uwe
>
> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>> @Uwe Did you have the time to run a backup of the home directory? I 
>> would
>> proceed and open an infra ticket if we are ready, I just need to know 
>> what "all
>> packages we
>> need for Jenkins Slaves" includes, as I am not sure how to look this 
>> up to
>> include it in the ticket.
>>
>> Christos
>>
>> Hi,
>>
>>
>>> I tried the do-release-upgrade on the older 18.04 machine. 
>>> Unfortunately
>>> it does not work, as there are too many PPA repositories in the config
>>> (nodejs,....) and those lead to package conflicts so do-release-upgrade
>>> bails out.
>>
>>> I could uninstall all those PPAs with "ppa-purge", but I don't know how
>>> to get puppet to reinstallthem after upgrade.
>>
>>> So it looks like we need to do the "rebuild node" with fresh OS 
>>> provided
>>> by ASF. That's the problem of nailed down instances with horrible 
>>> puppet
>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>> customers, it always causes issues administering machines for Solr
>>> installations!).
>>
>>> I can do the jenkins home directory backup tomorrow and then we can ask
>> them to reinstall newer OS. We just need to make sure to tell them that
>>> they should use a puppet configuration that contains all packages we
>> need for Jenkins Slaves.
>>
>>
>>> Uwe
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

go ahead. I preserved some config files. Actually the Gradle builds are 
easier to maintain as Ant, so basically it is just 2 files.

Uwe

Am 02.04.2025 um 17:21 schrieb Uwe Schindler:
> Oh sorry, will do! I ping you.
>
> Uwe
>
> Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
>> @Uwe Did you have the time to run a backup of the home directory? I 
>> would
>> proceed and open an infra ticket if we are ready, I just need to know 
>> what "all
>> packages we
>> need for Jenkins Slaves" includes, as I am not sure how to look this 
>> up to
>> include it in the ticket.
>>
>> Christos
>>
>> Hi,
>>
>>
>>> I tried the do-release-upgrade on the older 18.04 machine. 
>>> Unfortunately
>>> it does not work, as there are too many PPA repositories in the config
>>> (nodejs,....) and those lead to package conflicts so do-release-upgrade
>>> bails out.
>>
>>> I could uninstall all those PPAs with "ppa-purge", but I don't know how
>>> to get puppet to reinstallthem after upgrade.
>>
>>> So it looks like we need to do the "rebuild node" with fresh OS 
>>> provided
>>> by ASF. That's the problem of nailed down instances with horrible 
>>> puppet
>>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>>> customers, it always causes issues administering machines for Solr
>>> installations!).
>>
>>> I can do the jenkins home directory backup tomorrow and then we can ask
>> them to reinstall newer OS. We just need to make sure to tell them that
>>> they should use a puppet configuration that contains all packages we
>> need for Jenkins Slaves.
>>
>>
>>> Uwe
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Oh sorry, will do! I ping you.

Uwe

Am 02.04.2025 um 17:02 schrieb Christos Malliaridis:
> @Uwe Did you have the time to run a backup of the home directory? I would
> proceed and open an infra ticket if we are ready, I just need to know what "all
> packages we
> need for Jenkins Slaves" includes, as I am not sure how to look this up to
> include it in the ticket.
>
> Christos
>
> Hi,
>
>
>> I tried the do-release-upgrade on the older 18.04 machine. Unfortunately
>> it does not work, as there are too many PPA repositories in the config
>> (nodejs,....) and those lead to package conflicts so do-release-upgrade
>> bails out.
>
>> I could uninstall all those PPAs with "ppa-purge", but I don't know how
>> to get puppet to reinstallthem after upgrade.
>
>> So it looks like we need to do the "rebuild node" with fresh OS provided
>> by ASF. That's the problem of nailed down instances with horrible puppet
>> (sorry, but puppet is the worst tool that I try to avoid at all of my
>> customers, it always causes issues administering machines for Solr
>> installations!).
>
>> I can do the jenkins home directory backup tomorrow and then we can ask
> them to reinstall newer OS. We just need to make sure to tell them that
>> they should use a puppet configuration that contains all packages we
> need for Jenkins Slaves.
>
>
>> Uwe

-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
@Uwe Did you have the time to run a backup of the home directory? I would
proceed and open an infra ticket if we are ready, I just need to know what "all
packages we
need for Jenkins Slaves" includes, as I am not sure how to look this up to
include it in the ticket.

Christos

Hi,


> I tried the do-release-upgrade on the older 18.04 machine. Unfortunately
> it does not work, as there are too many PPA repositories in the config
> (nodejs,....) and those lead to package conflicts so do-release-upgrade
> bails out.


> I could uninstall all those PPAs with "ppa-purge", but I don't know how
> to get puppet to reinstallthem after upgrade.


> So it looks like we need to do the "rebuild node" with fresh OS provided
> by ASF. That's the problem of nailed down instances with horrible puppet
> (sorry, but puppet is the worst tool that I try to avoid at all of my
> customers, it always causes issues administering machines for Solr
> installations!).


> I can do the jenkins home directory backup tomorrow and then we can ask

them to reinstall newer OS. We just need to make sure to tell them that
> they should use a puppet configuration that contains all packages we

need for Jenkins Slaves.


> Uwe

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
@Uwe Did you have the time to run a backup of the home directory? I would
proceed and open an infra ticket if we are ready, I just need to know what "all
packages we
need for Jenkins Slaves" includes, as I am not sure how to look this up to
include it in the ticket.

Christos

Hi,


> I tried the do-release-upgrade on the older 18.04 machine. Unfortunately
> it does not work, as there are too many PPA repositories in the config
> (nodejs,....) and those lead to package conflicts so do-release-upgrade
> bails out.


> I could uninstall all those PPAs with "ppa-purge", but I don't know how
> to get puppet to reinstallthem after upgrade.


> So it looks like we need to do the "rebuild node" with fresh OS provided
> by ASF. That's the problem of nailed down instances with horrible puppet
> (sorry, but puppet is the worst tool that I try to avoid at all of my
> customers, it always causes issues administering machines for Solr
> installations!).


> I can do the jenkins home directory backup tomorrow and then we can ask

them to reinstall newer OS. We just need to make sure to tell them that
> they should use a puppet configuration that contains all packages we

need for Jenkins Slaves.


> Uwe

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

I tried the do-release-upgrade on the older 18.04 machine. Unfortunately 
it does not work, as there are too many PPA repositories in the config 
(nodejs,....) and those lead to package conflicts so do-release-upgrade 
bails out.

I could uninstall all those PPAs with "ppa-purge", but I don't know how 
to get puppet to reinstallthem after upgrade.

So it looks like we need to do the "rebuild node" with fresh OS provided 
by ASF. That's the problem of nailed down instances with horrible puppet 
(sorry, but puppet is the worst tool that I try to avoid at all of my 
customers, it always causes issues administering machines for Solr 
installations!).

I can do the jenkins home directory backup tomorrow and then we can ask 
them to reinstall newer OS. We just need to make sure to tell them that 
they should use a puppet configuration that contains all packages we 
need for Jenkins Slaves.

Uwe

Am 28.03.2025 um 23:02 schrieb Christos Malliaridis:
> The do-release-upgrade sounds like the way to go then.
>
> If you are willing and have the time to upgrade the machines as you 
> recommended, I would love to join you, if possible, so that I have 
> more insights about these processes. Personally, if I would have to 
> upgrade the machines myself, I would just follow the guides from the 
> internet.
>
> Best,
> Christos
>
> On Thu, 27 Mar 2025, 17:33 Uwe Schindler, <uw...@thetaphi.de> wrote:
>
>     Small update: both machines already use netplan and not old
>     interfaces
>     file, so do-release-upgrade is a no-brainer.
>
>     Uwe
>
>     Am 27.03.2025 um 16:23 schrieb Uwe Schindler:
>     >
>     > Hi,
>     >
>     > I have root access on both machines. Indeed one of them
>     (lucene-solr1)
>     > is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is
>     > Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first
>     one
>     > is out of support as far as I remember.
>     >
>     > There are two ways:
>     >
>     >   * do-release-upgrade => this works well on Ubuntu LTS nodes, never
>     >     had a problem with that. In fact the one on Ubuntu 20.04 was
>     >     updated like this (as far as I remember). The big downside: It
>     >     takes lots of time, as we need for the older machine 3
>     rounds and
>     >     reboots (and possibly having netplan issues) and 2 rounds
>     for the
>     >     newer one. I can do this, if needed. The pros are that we
>     can for
>     >     sure use our instances after that without Infra changing
>     Jenkins.
>     >     Infra still needs to be pinged afterwards to update puppet
>     version
>     >     (this was how it was done last time).
>     >   * reinstall everthing => this has the problem that there were
>     >     changes on the nodes which might be done in the jenkins home
>     >     directory. If we do this (and that's what you are suggesting),
>     >     please make a "tar cf" of the jenkins user home directory to
>     make
>     >     sure all config files in ".gradle" and the home dir are
>     copied. Of
>     >     course before doing this, do a "rm-rf" in the
>     >     "jenkins-slave/workspace" folder. The config files in the root
>     >     directory are there to instruct gradle how many cpus should be
>     >     used for the build to max out the hardware. The gradle default
>     >     installed by our gradle build do not max out the CPUs to allow
>     >     development in parallel. So it is important to copy
>     >     lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
>     >     .gradle/gradle.properties to the new node. Also make sure to
>     do a
>     >     list of all dpkg packages, so we don't forget something we might
>     >     need. With my recent experience, the defaults should be
>     enough (we
>     >     no longer build with crazy tools).
>     >
>     > Personally I am not a big fan of "reinstall OS because something
>     does
>     > not work". This has the side effect of a lot of time of non-working
>     > builds because some stupid detail is missing or INFRA needs to
>     install
>     > some of their custom Java packages over and over. With the usual
>     > round-trip-times of Infra this takes days.
>     >
>     > Some story on my own Jenkins server: I had to do a
>     reinstallation with
>     > the macOS Jenkins Slave on Policeman Jenkins but it costed me
>     half of
>     > a day to get the builds running again (OK, macOS is harder, but
>     > reinstalling missing stuff, setting up SSH keys,... is a stupid
>     task).
>     > Transplanting the Windows node (that was easy) and also the whole
>     > server from old hardware to a new hardware was much faster (1.5 hrs
>     > where most of the time was waiting for the rsync in recovery
>     console
>     > and updating EFI build, actually there ways no setup necessary
>     at all
>     > - it booted straight out of box and Jenkins autostarted and went
>     back
>     > to work).
>     >
>     > Important info: The setup of the nodes inside Jenkins can only
>     be done
>     > by Infra, we have no access. Generally it is better to not
>     change to
>     > much on the nodes and just upgrade them (first point above).
>     >
>     > Uwe
>     >
>     > Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
>     >> Hello everyone,
>     >>
>     >> In consultation with David Smiley, I am planning to request an
>     OS upgrade
>     >> for our build nodes *lucene-solr-1* and *lucene-solr-2* on
>     Monday, March
>     >> 31, as they are running older operating systems and causing
>     OS-related
>     >> build errors in the Solr project.
>     >>
>     >> Since the nodes will be replaced with new instances, they will
>     need to be
>     >> reconfigured for both projects. Please speak up if this may be
>     an issue or
>     >> if I should wait longer before requesting an upgr
>     <https://www.google.com/maps/search/ger+before+requesting+an+upgr?entry=gmail&source=g>ade
>     from infra. I will
>     >> also need someone with access to each project to configure the
>     nodes again
>     <https://www.google.com/maps/search/to+configure+the+nodes+again?entry=gmail&source=g>,
>     >> as I do not currently have sufficient access.
>     >>
>     >> Best,
>     >> Christos
>     >>
>     > --
>     > Uwe Schindler
>     > Achterdiek 19, D-28357 Bremen
>     > https://www.thetaphi.de
>     > eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>
>     -- 
>     Uwe Schindler
>     Achterdiek 19, D-28357 Bremen
>     https://www.thetaphi.de
>     eMail:uwe@thetaphi.de <ma...@thetaphi.de>
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
The do-release-upgrade sounds like the way to go then.

If you are willing and have the time to upgrade the machines as you
recommended, I would love to join you, if possible, so that I have more
insights about these processes. Personally, if I would have to upgrade the
machines myself, I would just follow the guides from the internet.

Best,
Christos

On Thu, 27 Mar 2025, 17:33 Uwe Schindler, <uw...@thetaphi.de> wrote:

> Small update: both machines already use netplan and not old interfaces
> file, so do-release-upgrade is a no-brainer.
>
> Uwe
>
> Am 27.03.2025 um 16:23 schrieb Uwe Schindler:
> >
> > Hi,
> >
> > I have root access on both machines. Indeed one of them (lucene-solr1)
> > is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is
> > Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one
> > is out of support as far as I remember.
> >
> > There are two ways:
> >
> >   * do-release-upgrade => this works well on Ubuntu LTS nodes, never
> >     had a problem with that. In fact the one on Ubuntu 20.04 was
> >     updated like this (as far as I remember). The big downside: It
> >     takes lots of time, as we need for the older machine 3 rounds and
> >     reboots (and possibly having netplan issues) and 2 rounds for the
> >     newer one. I can do this, if needed. The pros are that we can for
> >     sure use our instances after that without Infra changing Jenkins.
> >     Infra still needs to be pinged afterwards to update puppet version
> >     (this was how it was done last time).
> >   * reinstall everthing => this has the problem that there were
> >     changes on the nodes which might be done in the jenkins home
> >     directory. If we do this (and that's what you are suggesting),
> >     please make a "tar cf" of the jenkins user home directory to make
> >     sure all config files in ".gradle" and the home dir are copied. Of
> >     course before doing this, do a "rm-rf" in the
> >     "jenkins-slave/workspace" folder. The config files in the root
> >     directory are there to instruct gradle how many cpus should be
> >     used for the build to max out the hardware. The gradle default
> >     installed by our gradle build do not max out the CPUs to allow
> >     development in parallel. So it is important to copy
> >     lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
> >     .gradle/gradle.properties to the new node. Also make sure to do a
> >     list of all dpkg packages, so we don't forget something we might
> >     need. With my recent experience, the defaults should be enough (we
> >     no longer build with crazy tools).
> >
> > Personally I am not a big fan of "reinstall OS because something does
> > not work". This has the side effect of a lot of time of non-working
> > builds because some stupid detail is missing or INFRA needs to install
> > some of their custom Java packages over and over. With the usual
> > round-trip-times of Infra this takes days.
> >
> > Some story on my own Jenkins server: I had to do a reinstallation with
> > the macOS Jenkins Slave on Policeman Jenkins but it costed me half of
> > a day to get the builds running again (OK, macOS is harder, but
> > reinstalling missing stuff, setting up SSH keys,... is a stupid task).
> > Transplanting the Windows node (that was easy) and also the whole
> > server from old hardware to a new hardware was much faster (1.5 hrs
> > where most of the time was waiting for the rsync in recovery console
> > and updating EFI build, actually there ways no setup necessary at all
> > - it booted straight out of box and Jenkins autostarted and went back
> > to work).
> >
> > Important info: The setup of the nodes inside Jenkins can only be done
> > by Infra, we have no access. Generally it is better to not change to
> > much on the nodes and just upgrade them (first point above).
> >
> > Uwe
> >
> > Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
> >> Hello everyone,
> >>
> >> In consultation with David Smiley, I am planning to request an OS
> upgrade
> >> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> >> 31, as they are running older operating systems and causing OS-related
> >> build errors in the Solr project.
> >>
> >> Since the nodes will be replaced with new instances, they will need to
> be
> >> reconfigured for both projects. Please speak up if this may be an issue
> or
> >> if I should wait longer before requesting an upgr
> <https://www.google.com/maps/search/ger+before+requesting+an+upgr?entry=gmail&source=g>ade
> from infra. I will
> >> also need someone with access to each project to configure the nodes
> again
> <https://www.google.com/maps/search/to+configure+the+nodes+again?entry=gmail&source=g>
> ,
> >> as I do not currently have sufficient access.
> >>
> >> Best,
> >> Christos
> >>
> > --
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail:uwe@thetaphi.de
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
The do-release-upgrade sounds like the way to go then.

If you are willing and have the time to upgrade the machines as you
recommended, I would love to join you, if possible, so that I have more
insights about these processes. Personally, if I would have to upgrade the
machines myself, I would just follow the guides from the internet.

Best,
Christos

On Thu, 27 Mar 2025, 17:33 Uwe Schindler, <uw...@thetaphi.de> wrote:

> Small update: both machines already use netplan and not old interfaces
> file, so do-release-upgrade is a no-brainer.
>
> Uwe
>
> Am 27.03.2025 um 16:23 schrieb Uwe Schindler:
> >
> > Hi,
> >
> > I have root access on both machines. Indeed one of them (lucene-solr1)
> > is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is
> > Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one
> > is out of support as far as I remember.
> >
> > There are two ways:
> >
> >   * do-release-upgrade => this works well on Ubuntu LTS nodes, never
> >     had a problem with that. In fact the one on Ubuntu 20.04 was
> >     updated like this (as far as I remember). The big downside: It
> >     takes lots of time, as we need for the older machine 3 rounds and
> >     reboots (and possibly having netplan issues) and 2 rounds for the
> >     newer one. I can do this, if needed. The pros are that we can for
> >     sure use our instances after that without Infra changing Jenkins.
> >     Infra still needs to be pinged afterwards to update puppet version
> >     (this was how it was done last time).
> >   * reinstall everthing => this has the problem that there were
> >     changes on the nodes which might be done in the jenkins home
> >     directory. If we do this (and that's what you are suggesting),
> >     please make a "tar cf" of the jenkins user home directory to make
> >     sure all config files in ".gradle" and the home dir are copied. Of
> >     course before doing this, do a "rm-rf" in the
> >     "jenkins-slave/workspace" folder. The config files in the root
> >     directory are there to instruct gradle how many cpus should be
> >     used for the build to max out the hardware. The gradle default
> >     installed by our gradle build do not max out the CPUs to allow
> >     development in parallel. So it is important to copy
> >     lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
> >     .gradle/gradle.properties to the new node. Also make sure to do a
> >     list of all dpkg packages, so we don't forget something we might
> >     need. With my recent experience, the defaults should be enough (we
> >     no longer build with crazy tools).
> >
> > Personally I am not a big fan of "reinstall OS because something does
> > not work". This has the side effect of a lot of time of non-working
> > builds because some stupid detail is missing or INFRA needs to install
> > some of their custom Java packages over and over. With the usual
> > round-trip-times of Infra this takes days.
> >
> > Some story on my own Jenkins server: I had to do a reinstallation with
> > the macOS Jenkins Slave on Policeman Jenkins but it costed me half of
> > a day to get the builds running again (OK, macOS is harder, but
> > reinstalling missing stuff, setting up SSH keys,... is a stupid task).
> > Transplanting the Windows node (that was easy) and also the whole
> > server from old hardware to a new hardware was much faster (1.5 hrs
> > where most of the time was waiting for the rsync in recovery console
> > and updating EFI build, actually there ways no setup necessary at all
> > - it booted straight out of box and Jenkins autostarted and went back
> > to work).
> >
> > Important info: The setup of the nodes inside Jenkins can only be done
> > by Infra, we have no access. Generally it is better to not change to
> > much on the nodes and just upgrade them (first point above).
> >
> > Uwe
> >
> > Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
> >> Hello everyone,
> >>
> >> In consultation with David Smiley, I am planning to request an OS
> upgrade
> >> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> >> 31, as they are running older operating systems and causing OS-related
> >> build errors in the Solr project.
> >>
> >> Since the nodes will be replaced with new instances, they will need to
> be
> >> reconfigured for both projects. Please speak up if this may be an issue
> or
> >> if I should wait longer before requesting an upgr
> <https://www.google.com/maps/search/ger+before+requesting+an+upgr?entry=gmail&source=g>ade
> from infra. I will
> >> also need someone with access to each project to configure the nodes
> again
> <https://www.google.com/maps/search/to+configure+the+nodes+again?entry=gmail&source=g>
> ,
> >> as I do not currently have sufficient access.
> >>
> >> Best,
> >> Christos
> >>
> > --
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail:uwe@thetaphi.de
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Small update: both machines already use netplan and not old interfaces 
file, so do-release-upgrade is a no-brainer.

Uwe

Am 27.03.2025 um 16:23 schrieb Uwe Schindler:
>
> Hi,
>
> I have root access on both machines. Indeed one of them (lucene-solr1) 
> is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is 
> Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one 
> is out of support as far as I remember.
>
> There are two ways:
>
>   * do-release-upgrade => this works well on Ubuntu LTS nodes, never
>     had a problem with that. In fact the one on Ubuntu 20.04 was
>     updated like this (as far as I remember). The big downside: It
>     takes lots of time, as we need for the older machine 3 rounds and
>     reboots (and possibly having netplan issues) and 2 rounds for the
>     newer one. I can do this, if needed. The pros are that we can for
>     sure use our instances after that without Infra changing Jenkins.
>     Infra still needs to be pinged afterwards to update puppet version
>     (this was how it was done last time).
>   * reinstall everthing => this has the problem that there were
>     changes on the nodes which might be done in the jenkins home
>     directory. If we do this (and that's what you are suggesting),
>     please make a "tar cf" of the jenkins user home directory to make
>     sure all config files in ".gradle" and the home dir are copied. Of
>     course before doing this, do a "rm-rf" in the
>     "jenkins-slave/workspace" folder. The config files in the root
>     directory are there to instruct gradle how many cpus should be
>     used for the build to max out the hardware. The gradle default
>     installed by our gradle build do not max out the CPUs to allow
>     development in parallel. So it is important to copy
>     lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
>     .gradle/gradle.properties to the new node. Also make sure to do a
>     list of all dpkg packages, so we don't forget something we might
>     need. With my recent experience, the defaults should be enough (we
>     no longer build with crazy tools).
>
> Personally I am not a big fan of "reinstall OS because something does 
> not work". This has the side effect of a lot of time of non-working 
> builds because some stupid detail is missing or INFRA needs to install 
> some of their custom Java packages over and over. With the usual 
> round-trip-times of Infra this takes days.
>
> Some story on my own Jenkins server: I had to do a reinstallation with 
> the macOS Jenkins Slave on Policeman Jenkins but it costed me half of 
> a day to get the builds running again (OK, macOS is harder, but 
> reinstalling missing stuff, setting up SSH keys,... is a stupid task). 
> Transplanting the Windows node (that was easy) and also the whole 
> server from old hardware to a new hardware was much faster (1.5 hrs 
> where most of the time was waiting for the rsync in recovery console 
> and updating EFI build, actually there ways no setup necessary at all 
> - it booted straight out of box and Jenkins autostarted and went back 
> to work).
>
> Important info: The setup of the nodes inside Jenkins can only be done 
> by Infra, we have no access. Generally it is better to not change to 
> much on the nodes and just upgrade them (first point above).
>
> Uwe
>
> Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
>> Hello everyone,
>>
>> In consultation with David Smiley, I am planning to request an OS upgrade
>> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
>> 31, as they are running older operating systems and causing OS-related
>> build errors in the Solr project.
>>
>> Since the nodes will be replaced with new instances, they will need to be
>> reconfigured for both projects. Please speak up if this may be an issue or
>> if I should wait longer before requesting an upgrade from infra. I will
>> also need someone with access to each project to configure the nodes again,
>> as I do not currently have sufficient access.
>>
>> Best,
>> Christos
>>
> -- 
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de

-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Small update: both machines already use netplan and not old interfaces 
file, so do-release-upgrade is a no-brainer.

Uwe

Am 27.03.2025 um 16:23 schrieb Uwe Schindler:
>
> Hi,
>
> I have root access on both machines. Indeed one of them (lucene-solr1) 
> is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is 
> Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one 
> is out of support as far as I remember.
>
> There are two ways:
>
>   * do-release-upgrade => this works well on Ubuntu LTS nodes, never
>     had a problem with that. In fact the one on Ubuntu 20.04 was
>     updated like this (as far as I remember). The big downside: It
>     takes lots of time, as we need for the older machine 3 rounds and
>     reboots (and possibly having netplan issues) and 2 rounds for the
>     newer one. I can do this, if needed. The pros are that we can for
>     sure use our instances after that without Infra changing Jenkins.
>     Infra still needs to be pinged afterwards to update puppet version
>     (this was how it was done last time).
>   * reinstall everthing => this has the problem that there were
>     changes on the nodes which might be done in the jenkins home
>     directory. If we do this (and that's what you are suggesting),
>     please make a "tar cf" of the jenkins user home directory to make
>     sure all config files in ".gradle" and the home dir are copied. Of
>     course before doing this, do a "rm-rf" in the
>     "jenkins-slave/workspace" folder. The config files in the root
>     directory are there to instruct gradle how many cpus should be
>     used for the build to max out the hardware. The gradle default
>     installed by our gradle build do not max out the CPUs to allow
>     development in parallel. So it is important to copy
>     lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
>     .gradle/gradle.properties to the new node. Also make sure to do a
>     list of all dpkg packages, so we don't forget something we might
>     need. With my recent experience, the defaults should be enough (we
>     no longer build with crazy tools).
>
> Personally I am not a big fan of "reinstall OS because something does 
> not work". This has the side effect of a lot of time of non-working 
> builds because some stupid detail is missing or INFRA needs to install 
> some of their custom Java packages over and over. With the usual 
> round-trip-times of Infra this takes days.
>
> Some story on my own Jenkins server: I had to do a reinstallation with 
> the macOS Jenkins Slave on Policeman Jenkins but it costed me half of 
> a day to get the builds running again (OK, macOS is harder, but 
> reinstalling missing stuff, setting up SSH keys,... is a stupid task). 
> Transplanting the Windows node (that was easy) and also the whole 
> server from old hardware to a new hardware was much faster (1.5 hrs 
> where most of the time was waiting for the rsync in recovery console 
> and updating EFI build, actually there ways no setup necessary at all 
> - it booted straight out of box and Jenkins autostarted and went back 
> to work).
>
> Important info: The setup of the nodes inside Jenkins can only be done 
> by Infra, we have no access. Generally it is better to not change to 
> much on the nodes and just upgrade them (first point above).
>
> Uwe
>
> Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
>> Hello everyone,
>>
>> In consultation with David Smiley, I am planning to request an OS upgrade
>> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
>> 31, as they are running older operating systems and causing OS-related
>> build errors in the Solr project.
>>
>> Since the nodes will be replaced with new instances, they will need to be
>> reconfigured for both projects. Please speak up if this may be an issue or
>> if I should wait longer before requesting an upgrade from infra. I will
>> also need someone with access to each project to configure the nodes again,
>> as I do not currently have sufficient access.
>>
>> Best,
>> Christos
>>
> -- 
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de

-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

I have root access on both machines. Indeed one of them (lucene-solr1) 
is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is 
Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one is 
out of support as far as I remember.

There are two ways:

  * do-release-upgrade => this works well on Ubuntu LTS nodes, never had
    a problem with that. In fact the one on Ubuntu 20.04 was updated
    like this (as far as I remember). The big downside: It takes lots of
    time, as we need for the older machine 3 rounds and reboots (and
    possibly having netplan issues) and 2 rounds for the newer one. I
    can do this, if needed. The pros are that we can for sure use our
    instances after that without Infra changing Jenkins. Infra still
    needs to be pinged afterwards to update puppet version (this was how
    it was done last time).
  * reinstall everthing => this has the problem that there were changes
    on the nodes which might be done in the jenkins home directory. If
    we do this (and that's what you are suggesting), please make a "tar
    cf" of the jenkins user home directory to make sure all config files
    in ".gradle" and the home dir are copied. Of course before doing
    this, do a "rm-rf" in the "jenkins-slave/workspace" folder. The
    config files in the root directory are there to instruct gradle how
    many cpus should be used for the build to max out the hardware. The
    gradle default installed by our gradle build do not max out the CPUs
    to allow development in parallel. So it is important to copy
    lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
    .gradle/gradle.properties to the new node. Also make sure to do a
    list of all dpkg packages, so we don't forget something we might
    need. With my recent experience, the defaults should be enough (we
    no longer build with crazy tools).

Personally I am not a big fan of "reinstall OS because something does 
not work". This has the side effect of a lot of time of non-working 
builds because some stupid detail is missing or INFRA needs to install 
some of their custom Java packages over and over. With the usual 
round-trip-times of Infra this takes days.

Some story on my own Jenkins server: I had to do a reinstallation with 
the macOS Jenkins Slave on Policeman Jenkins but it costed me half of a 
day to get the builds running again (OK, macOS is harder, but 
reinstalling missing stuff, setting up SSH keys,... is a stupid task). 
Transplanting the Windows node (that was easy) and also the whole server 
from old hardware to a new hardware was much faster (1.5 hrs where most 
of the time was waiting for the rsync in recovery console and updating 
EFI build, actually there ways no setup necessary at all - it booted 
straight out of box and Jenkins autostarted and went back to work).

Important info: The setup of the nodes inside Jenkins can only be done 
by Infra, we have no access. Generally it is better to not change to 
much on the nodes and just upgrade them (first point above).

Uwe

Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

I have root access on both machines. Indeed one of them (lucene-solr1) 
is Ubuntu 18.04 LTS and the other one (lucene-solr - no digit) is 
Unbuntu 20.04 LTS. In fact, both are outdated, whereas the first one is 
out of support as far as I remember.

There are two ways:

  * do-release-upgrade => this works well on Ubuntu LTS nodes, never had
    a problem with that. In fact the one on Ubuntu 20.04 was updated
    like this (as far as I remember). The big downside: It takes lots of
    time, as we need for the older machine 3 rounds and reboots (and
    possibly having netplan issues) and 2 rounds for the newer one. I
    can do this, if needed. The pros are that we can for sure use our
    instances after that without Infra changing Jenkins. Infra still
    needs to be pinged afterwards to update puppet version (this was how
    it was done last time).
  * reinstall everthing => this has the problem that there were changes
    on the nodes which might be done in the jenkins home directory. If
    we do this (and that's what you are suggesting), please make a "tar
    cf" of the jenkins user home directory to make sure all config files
    in ".gradle" and the home dir are copied. Of course before doing
    this, do a "rm-rf" in the "jenkins-slave/workspace" folder. The
    config files in the root directory are there to instruct gradle how
    many cpus should be used for the build to max out the hardware. The
    gradle default installed by our gradle build do not max out the CPUs
    to allow development in parallel. So it is important to copy
    lucene.build.properties (for Lucene/Solr 8x. legacy builds) and
    .gradle/gradle.properties to the new node. Also make sure to do a
    list of all dpkg packages, so we don't forget something we might
    need. With my recent experience, the defaults should be enough (we
    no longer build with crazy tools).

Personally I am not a big fan of "reinstall OS because something does 
not work". This has the side effect of a lot of time of non-working 
builds because some stupid detail is missing or INFRA needs to install 
some of their custom Java packages over and over. With the usual 
round-trip-times of Infra this takes days.

Some story on my own Jenkins server: I had to do a reinstallation with 
the macOS Jenkins Slave on Policeman Jenkins but it costed me half of a 
day to get the builds running again (OK, macOS is harder, but 
reinstalling missing stuff, setting up SSH keys,... is a stupid task). 
Transplanting the Windows node (that was easy) and also the whole server 
from old hardware to a new hardware was much faster (1.5 hrs where most 
of the time was waiting for the rsync in recovery console and updating 
EFI build, actually there ways no setup necessary at all - it booted 
straight out of box and Jenkins autostarted and went back to work).

Important info: The setup of the nodes inside Jenkins can only be done 
by Infra, we have no access. Generally it is better to not change to 
much on the nodes and just upgrade them (first point above).

Uwe

Am 26.03.2025 um 18:04 schrieb Christos Malliaridis:
> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
I do have access to those machines and I can help, but a plan of what needs
to be done and in which order would be useful. :)

Dawid

On Wed, Mar 26, 2025 at 6:05 PM Christos Malliaridis <ma...@apache.org>
wrote:

> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Security updates ongoing... reboot ongoing... running again... 😉

Uwe

Am 27.03.2025 um 16:40 schrieb Uwe Schindler:
> Hi,
>
> see my mails on the lucene ML, these ones were lost as they only got 
> to Solr.
>
> Actually, I was involved together with Jenkins and I also have root 
> access. Yes, they were provided by Infra, but there was some 
> additional work needed to make them Jenkins nodes. It is long ago that 
> I touched those nodes.
>
> In the meantime I triggered the normal "apt" updates a minute ago, but 
> they were already almost uptodate. They still need a reboot to get 
> kernel patches for safety. Will do this once they are "free"!
>
> Uwe
>
> Am 26.03.2025 um 20:22 schrieb Dawid Weiss:
>>> That is indeed helpful, I am just not familiar enough with the node 
>>> setup
>>> to provide that kind of information.
>>>
>> None of us do I think. I believe both these boxes were provisioned by 
>> infra
>> but I have no idea how they were set up
>> (manually or via some sort of automation). I would think it's a
>> puppet-style automation that is in place here but I've no
>> idea, really.
>>
>> I think opening an infra issue in jira would be a good start to 
>> discuss how
>> to proceed with updating these boxes. I have
>> a feeling we can't do it on our own - it'll require infra's assistance.
>>
>> D.
>>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Security updates ongoing... reboot ongoing... running again... 😉

Uwe

Am 27.03.2025 um 16:40 schrieb Uwe Schindler:
> Hi,
>
> see my mails on the lucene ML, these ones were lost as they only got 
> to Solr.
>
> Actually, I was involved together with Jenkins and I also have root 
> access. Yes, they were provided by Infra, but there was some 
> additional work needed to make them Jenkins nodes. It is long ago that 
> I touched those nodes.
>
> In the meantime I triggered the normal "apt" updates a minute ago, but 
> they were already almost uptodate. They still need a reboot to get 
> kernel patches for safety. Will do this once they are "free"!
>
> Uwe
>
> Am 26.03.2025 um 20:22 schrieb Dawid Weiss:
>>> That is indeed helpful, I am just not familiar enough with the node 
>>> setup
>>> to provide that kind of information.
>>>
>> None of us do I think. I believe both these boxes were provisioned by 
>> infra
>> but I have no idea how they were set up
>> (manually or via some sort of automation). I would think it's a
>> puppet-style automation that is in place here but I've no
>> idea, really.
>>
>> I think opening an infra issue in jira would be a good start to 
>> discuss how
>> to proceed with updating these boxes. I have
>> a feeling we can't do it on our own - it'll require infra's assistance.
>>
>> D.
>>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

see my mails on the lucene ML, these ones were lost as they only got to 
Solr.

Actually, I was involved together with Jenkins and I also have root 
access. Yes, they were provided by Infra, but there was some additional 
work needed to make them Jenkins nodes. It is long ago that I touched 
those nodes.

In the meantime I triggered the normal "apt" updates a minute ago, but 
they were already almost uptodate. They still need a reboot to get 
kernel patches for safety. Will do this once they are "free"!

Uwe

Am 26.03.2025 um 20:22 schrieb Dawid Weiss:
>> That is indeed helpful, I am just not familiar enough with the node setup
>> to provide that kind of information.
>>
> None of us do I think. I believe both these boxes were provisioned by infra
> but I have no idea how they were set up
> (manually or via some sort of automation). I would think it's a
> puppet-style automation that is in place here but I've no
> idea, really.
>
> I think opening an infra issue in jira would be a good start to discuss how
> to proceed with updating these boxes. I have
> a feeling we can't do it on our own - it'll require infra's assistance.
>
> D.
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

see my mails on the lucene ML, these ones were lost as they only got to 
Solr.

Actually, I was involved together with Jenkins and I also have root 
access. Yes, they were provided by Infra, but there was some additional 
work needed to make them Jenkins nodes. It is long ago that I touched 
those nodes.

In the meantime I triggered the normal "apt" updates a minute ago, but 
they were already almost uptodate. They still need a reboot to get 
kernel patches for safety. Will do this once they are "free"!

Uwe

Am 26.03.2025 um 20:22 schrieb Dawid Weiss:
>> That is indeed helpful, I am just not familiar enough with the node setup
>> to provide that kind of information.
>>
> None of us do I think. I believe both these boxes were provisioned by infra
> but I have no idea how they were set up
> (manually or via some sort of automation). I would think it's a
> puppet-style automation that is in place here but I've no
> idea, really.
>
> I think opening an infra issue in jira would be a good start to discuss how
> to proceed with updating these boxes. I have
> a feeling we can't do it on our own - it'll require infra's assistance.
>
> D.
>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org


Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
>
> That is indeed helpful, I am just not familiar enough with the node setup
> to provide that kind of information.
>

None of us do I think. I believe both these boxes were provisioned by infra
but I have no idea how they were set up
(manually or via some sort of automation). I would think it's a
puppet-style automation that is in place here but I've no
idea, really.

I think opening an infra issue in jira would be a good start to discuss how
to proceed with updating these boxes. I have
a feeling we can't do it on our own - it'll require infra's assistance.

D.

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Christos Malliaridis <c....@gmail.com>.
>
> Was it actually only one machine that was too out-of-date?


lucene-solr-1 is too out-of-date to cause an error, yes. lucene-solr-2 LTS
reaches EOL at the end of this year (?). So I think upgrading both is the
way to go.

a plan of what needs to be done and in which order would be useful. :)


That is indeed helpful, I am just not familiar enough with the node setup
to provide that kind of information.

*P.S. If you have access to the node system information, you may check the
os.version and get the EOL dates from https://endoflife.date/linux
<https://endoflife.date/linux>*

On Wed, Mar 26, 2025 at 8:36 PM David Smiley <ds...@apache.org> wrote:

> Was it actually only one machine that was too out-of-date?
>
> On Wed, Mar 26, 2025 at 1:05 PM Christos Malliaridis <
> malliaridis@apache.org>
> wrote:
>
> > Hello everyone,
> >
> > In consultation with David Smiley, I am planning to request an OS upgrade
> > for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> > 31, as they are running older operating systems and causing OS-related
> > build errors in the Solr project.
> >
> > Since the nodes will be replaced with new instances, they will need to be
> > reconfigured for both projects. Please speak up if this may be an issue
> or
> > if I should wait longer before requesting an upgrade from infra. I will
> > also need someone with access to each project to configure the nodes
> again,
> > as I do not currently have sufficient access.
> >
> > Best,
> > Christos
> >
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by David Smiley <ds...@apache.org>.
Was it actually only one machine that was too out-of-date?

On Wed, Mar 26, 2025 at 1:05 PM Christos Malliaridis <ma...@apache.org>
wrote:

> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by Dawid Weiss <da...@gmail.com>.
I do have access to those machines and I can help, but a plan of what needs
to be done and in which order would be useful. :)

Dawid

On Wed, Mar 26, 2025 at 6:05 PM Christos Malliaridis <ma...@apache.org>
wrote:

> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>

Re: Planned OS Upgrade for lucene-solr-1 and lucene-solr-2

Posted by David Smiley <ds...@apache.org>.
Was it actually only one machine that was too out-of-date?

On Wed, Mar 26, 2025 at 1:05 PM Christos Malliaridis <ma...@apache.org>
wrote:

> Hello everyone,
>
> In consultation with David Smiley, I am planning to request an OS upgrade
> for our build nodes *lucene-solr-1* and *lucene-solr-2* on Monday, March
> 31, as they are running older operating systems and causing OS-related
> build errors in the Solr project.
>
> Since the nodes will be replaced with new instances, they will need to be
> reconfigured for both projects. Please speak up if this may be an issue or
> if I should wait longer before requesting an upgrade from infra. I will
> also need someone with access to each project to configure the nodes again,
> as I do not currently have sufficient access.
>
> Best,
> Christos
>