You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@bigtop.apache.org by Luca Toscano <to...@gmail.com> on 2021/11/19 07:15:16 UTC

Set up CI for Debian 11 Bullseye

Hi everybody!

I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
IIUC I'd need to add the CI jobs now, but of course I don't have
permissions. If anybody has some time to help I'd be happy to learn
the process and help out :)

Thanks!

Luca

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Update:

I have deleted old build files from the Jenkins UI, mostly related to
old Bigtop releases (1.4/1.5), keeping only the last 3. The Jenkins
master has now ~470GB of free disk space, that is more than enough to
keep building the current projects without incurring in disk space
issues.

Question: should we delete all old builds, keeping only data for the
projects that are actually being used (like Bigtop 3.x, trunk, etc..)
? Is there any value in keeping logs/etc.. about what has been done in
the past? What I mean is if we have use cases that need us to go back
in time to inspect what was done :)

Thanks!

Luca

On Mon, Dec 6, 2021 at 9:57 PM Luca Toscano <to...@gmail.com> wrote:
>
> Update: I have deleted (via the UI) the failed builds that I tried in
> these days for Trunk packages, and I freed ~100G. Is there any
> specific reason to keep more than 2/3 builds for each project by
> default?
>
> The idea would be to do something like:
> https://plugins.jenkins.io/build-discarder/#plugin-content-getting-started
>
> My understanding is that old builds are kept for no real reason, all
> packages are stored in the Apache archive and we don't really need to
> keep build logs/config around. If nobody opposes it, I'd add a basic
> global config to keep 3 builds for each project and discard the rest.
>
> Let me know what you think!
>
> Luca
>
>
> Luca
>
> On Sun, Dec 5, 2021 at 10:02 AM Luca Toscano <to...@gmail.com> wrote:
> >
> > Thanks a lot Olaf, totally missed the credentials for AWS, I was able
> > to get the new hostnames and ssh in. The master's root partition seems
> > to be full afaics, and under /home/jenkins/jobs there is a ton of old
> > build data (even from Bigtop 1.1.0 etc..). I am not very expert in
> > jenkins but I am wondering if a Global build discarder (found in
> > https://ci.bigtop.apache.org/configure) could be something worth
> > testing. Was it done in the past? Is there any downside in using it?
> > For example, we could keep the last 6 months (or any time window)
> > worth of build data. Let me know what you think!
> >
> > Luca
> >
> > On Sat, Dec 4, 2021 at 9:08 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >
> > > Hi Luca,
> > >
> > > the name changed. There is now some space available.
> > >
> > > Check with the EC2 console to see the current public IP adresses .
> > > https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running
> > >
> > > Best
> > >   Olaf
> > >
> > > > Am 04.12.2021 um 18:23 schrieb Luca Toscano <to...@gmail.com>:
> > > >
> > > > Hi :)
> > > >
> > > > https://ci.bigtop.apache.org/log/all seems to indicate that we are
> > > > again in trouble, docker-slave-02 seems full. I got the credentials to
> > > > access the node but I can reach it (my ssh session hangs
> > > > indefinitely), is there anybody that can take a look?
> > > >
> > > > Thanks,
> > > >
> > > > Luca
> > > >
> > > > On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
> > > >> I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk.
> > > >> I removed all bug the last ~3000 job dirs.
> > > >>
> > > >> Can someone please look at the config of „bigpetstore“, please?
> > > >>
> > > >> At least we have some air to breath now again on jenkins.
> > > >>
> > > >> Olaf
> > > >>
> > > >>
> > > >>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
> > > >>>
> > > >>> Regarding setup of jenkins, you can find a lot of info here:
> > > >>>
> > > >>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
> > > >>>
> > > >>> Olaf
> > > >>>
> > > >>>
> > > >>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
> > > >>>>
> > > >>>> I am working with luca to get him access ..
> > > >>>>
> > > >>>> Olaf
> > > >>>>
> > > >>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > > >>>>>
> > > >>>>> Keep reporting issues, sorry :)
> > > >>>>>
> > > >>>>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
> > > >>>>> disk space issues on the Jenkins node, builds are stopped for the
> > > >>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I
> > > >>>>> guess it means accessing to the VM/hosts that are running Jenkins and
> > > >>>>> I have never done it (not even sure where to put my ssh key etc, what
> > > >>>>> is the hostname, etc..).
> > > >>>>>
> > > >>>>> Thanks!
> > > >>>>>
> > > >>>>> Luca
> > > >>>>>
> > > >>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > > >>>>>>
> > > >>>>>> Hi Luca,
> > > >>>>>>
> > > >>>>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
> > > >>>>>>
> > > >>>>>> Olaf
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > > >>>>>>>
> > > >>>>>>> Hi Olaf,
> > > >>>>>>>
> > > >>>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
> > > >>>>>>> is valid though (http redirects to https), but I don't get why we
> > > >>>>>>> should delete the line (also the code returns a HTTP 201 afaics so it
> > > >>>>>>> seems not failing to find it).
> > > >>>>>>>
> > > >>>>>>> Lemme know, I am probably missing something, too many new things :)
> > > >>>>>>>
> > > >>>>>>> Luca
> > > >>>>>>>
> > > >>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> Hi Luca,
> > > >>>>>>>>
> > > >>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
> > > >>>>>>>>
> > > >>>>>>>> Try to delete this line:
> > > >>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
> > > >>>>>>>>
> > > >>>>>>>> Olaf
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > > >>>>>>>>>
> > > >>>>>>>>> Hi Olaf,
> > > >>>>>>>>>
> > > >>>>>>>>> I restarted the jobs and some packages started to build, thanks! I
> > > >>>>>>>>> found another weird error though:
> > > >>>>>>>>>
> > > >>>>>>>>> """
> > > >>>>>>>>> * What went wrong:
> > > >>>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
> > > >>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
> > > >>>>>>>>> """
> > > >>>>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
> > > >>>>>>>>>
> > > >>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
> > > >>>>>>>>>
> > > >>>>>>>>> Luca
> > > >>>>>>>>>
> > > >>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hi Luca,
> > > >>>>>>>>>>
> > > >>>>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> > > >>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> If this network is not present you might get this kind of error
> > > >>>>>>>>>> So try to restart nexus.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Best
> > > >>>>>>>>>> Olaf
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Update:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
> > > >>>>>>>>>>> periodic execution.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
> > > >>>>>>>>>>> builds were failing for the same error:
> > > >>>>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> > > >>>>>>>>>>> docker: Error response from daemon: cannot join network of a non
> > > >>>>>>>>>>> running container
> > > >>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I have probably missed something, I don't exactly understand the error
> > > >>>>>>>>>>> so any help would be appreciated :)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Luca
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> > > >>>>>>>>>>>> package build. I can also enable periodic execution but I am not sure
> > > >>>>>>>>>>>> what I should put in the related time interval field (I didn't find it
> > > >>>>>>>>>>>> in the docs).
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Luca
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hi Olaf! Thanks a lot :)
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
> > > >>>>>>>>>>>>> Ubuntu that seems unrelated, see
> > > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
> > > >>>>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
> > > >>>>>>>>>>>>> 11 images published to Dockerhub.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> IIUC the last step is to modify
> > > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
> > > >>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> > > >>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
> > > >>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Luca
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi Luca,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I gave you super powers.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Best
> > > >>>>>>>>>>>>>> Olaf
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi everybody!
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> > > >>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> > > >>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> > > >>>>>>>>>>>>>>> the process and help out :)
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Luca
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Update: I have deleted (via the UI) the failed builds that I tried in
these days for Trunk packages, and I freed ~100G. Is there any
specific reason to keep more than 2/3 builds for each project by
default?

The idea would be to do something like:
https://plugins.jenkins.io/build-discarder/#plugin-content-getting-started

My understanding is that old builds are kept for no real reason, all
packages are stored in the Apache archive and we don't really need to
keep build logs/config around. If nobody opposes it, I'd add a basic
global config to keep 3 builds for each project and discard the rest.

Let me know what you think!

Luca


Luca

On Sun, Dec 5, 2021 at 10:02 AM Luca Toscano <to...@gmail.com> wrote:
>
> Thanks a lot Olaf, totally missed the credentials for AWS, I was able
> to get the new hostnames and ssh in. The master's root partition seems
> to be full afaics, and under /home/jenkins/jobs there is a ton of old
> build data (even from Bigtop 1.1.0 etc..). I am not very expert in
> jenkins but I am wondering if a Global build discarder (found in
> https://ci.bigtop.apache.org/configure) could be something worth
> testing. Was it done in the past? Is there any downside in using it?
> For example, we could keep the last 6 months (or any time window)
> worth of build data. Let me know what you think!
>
> Luca
>
> On Sat, Dec 4, 2021 at 9:08 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >
> > Hi Luca,
> >
> > the name changed. There is now some space available.
> >
> > Check with the EC2 console to see the current public IP adresses .
> > https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running
> >
> > Best
> >   Olaf
> >
> > > Am 04.12.2021 um 18:23 schrieb Luca Toscano <to...@gmail.com>:
> > >
> > > Hi :)
> > >
> > > https://ci.bigtop.apache.org/log/all seems to indicate that we are
> > > again in trouble, docker-slave-02 seems full. I got the credentials to
> > > access the node but I can reach it (my ssh session hangs
> > > indefinitely), is there anybody that can take a look?
> > >
> > > Thanks,
> > >
> > > Luca
> > >
> > > On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >>
> > >> Hi,
> > >>
> > >> I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
> > >> I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk.
> > >> I removed all bug the last ~3000 job dirs.
> > >>
> > >> Can someone please look at the config of „bigpetstore“, please?
> > >>
> > >> At least we have some air to breath now again on jenkins.
> > >>
> > >> Olaf
> > >>
> > >>
> > >>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
> > >>>
> > >>> Regarding setup of jenkins, you can find a lot of info here:
> > >>>
> > >>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
> > >>>
> > >>> Olaf
> > >>>
> > >>>
> > >>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
> > >>>>
> > >>>> I am working with luca to get him access ..
> > >>>>
> > >>>> Olaf
> > >>>>
> > >>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > >>>>>
> > >>>>> Keep reporting issues, sorry :)
> > >>>>>
> > >>>>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
> > >>>>> disk space issues on the Jenkins node, builds are stopped for the
> > >>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I
> > >>>>> guess it means accessing to the VM/hosts that are running Jenkins and
> > >>>>> I have never done it (not even sure where to put my ssh key etc, what
> > >>>>> is the hostname, etc..).
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>> Luca
> > >>>>>
> > >>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > >>>>>>
> > >>>>>> Hi Luca,
> > >>>>>>
> > >>>>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
> > >>>>>>
> > >>>>>> Olaf
> > >>>>>>
> > >>>>>>
> > >>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > >>>>>>>
> > >>>>>>> Hi Olaf,
> > >>>>>>>
> > >>>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
> > >>>>>>> is valid though (http redirects to https), but I don't get why we
> > >>>>>>> should delete the line (also the code returns a HTTP 201 afaics so it
> > >>>>>>> seems not failing to find it).
> > >>>>>>>
> > >>>>>>> Lemme know, I am probably missing something, too many new things :)
> > >>>>>>>
> > >>>>>>> Luca
> > >>>>>>>
> > >>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > >>>>>>>>
> > >>>>>>>> Hi Luca,
> > >>>>>>>>
> > >>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
> > >>>>>>>>
> > >>>>>>>> Try to delete this line:
> > >>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
> > >>>>>>>>
> > >>>>>>>> Olaf
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > >>>>>>>>>
> > >>>>>>>>> Hi Olaf,
> > >>>>>>>>>
> > >>>>>>>>> I restarted the jobs and some packages started to build, thanks! I
> > >>>>>>>>> found another weird error though:
> > >>>>>>>>>
> > >>>>>>>>> """
> > >>>>>>>>> * What went wrong:
> > >>>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
> > >>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
> > >>>>>>>>> """
> > >>>>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
> > >>>>>>>>>
> > >>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
> > >>>>>>>>>
> > >>>>>>>>> Luca
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Luca,
> > >>>>>>>>>>
> > >>>>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> > >>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
> > >>>>>>>>>>
> > >>>>>>>>>> If this network is not present you might get this kind of error
> > >>>>>>>>>> So try to restart nexus.
> > >>>>>>>>>>
> > >>>>>>>>>> Best
> > >>>>>>>>>> Olaf
> > >>>>>>>>>>
> > >>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Update:
> > >>>>>>>>>>>
> > >>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
> > >>>>>>>>>>> periodic execution.
> > >>>>>>>>>>>
> > >>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
> > >>>>>>>>>>> builds were failing for the same error:
> > >>>>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> > >>>>>>>>>>> docker: Error response from daemon: cannot join network of a non
> > >>>>>>>>>>> running container
> > >>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have probably missed something, I don't exactly understand the error
> > >>>>>>>>>>> so any help would be appreciated :)
> > >>>>>>>>>>>
> > >>>>>>>>>>> Luca
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> > >>>>>>>>>>>> package build. I can also enable periodic execution but I am not sure
> > >>>>>>>>>>>> what I should put in the related time interval field (I didn't find it
> > >>>>>>>>>>>> in the docs).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Luca
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Olaf! Thanks a lot :)
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
> > >>>>>>>>>>>>> Ubuntu that seems unrelated, see
> > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
> > >>>>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
> > >>>>>>>>>>>>> 11 images published to Dockerhub.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> IIUC the last step is to modify
> > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
> > >>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> > >>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
> > >>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Luca
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi Luca,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I gave you super powers.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best
> > >>>>>>>>>>>>>> Olaf
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi everybody!
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> > >>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> > >>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> > >>>>>>>>>>>>>>> the process and help out :)
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Luca
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Thanks a lot Olaf, totally missed the credentials for AWS, I was able
to get the new hostnames and ssh in. The master's root partition seems
to be full afaics, and under /home/jenkins/jobs there is a ton of old
build data (even from Bigtop 1.1.0 etc..). I am not very expert in
jenkins but I am wondering if a Global build discarder (found in
https://ci.bigtop.apache.org/configure) could be something worth
testing. Was it done in the past? Is there any downside in using it?
For example, we could keep the last 6 months (or any time window)
worth of build data. Let me know what you think!

Luca

On Sat, Dec 4, 2021 at 9:08 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Luca,
>
> the name changed. There is now some space available.
>
> Check with the EC2 console to see the current public IP adresses .
> https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running
>
> Best
>   Olaf
>
> > Am 04.12.2021 um 18:23 schrieb Luca Toscano <to...@gmail.com>:
> >
> > Hi :)
> >
> > https://ci.bigtop.apache.org/log/all seems to indicate that we are
> > again in trouble, docker-slave-02 seems full. I got the credentials to
> > access the node but I can reach it (my ssh session hangs
> > indefinitely), is there anybody that can take a look?
> >
> > Thanks,
> >
> > Luca
> >
> > On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>
> >> Hi,
> >>
> >> I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
> >> I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk.
> >> I removed all bug the last ~3000 job dirs.
> >>
> >> Can someone please look at the config of „bigpetstore“, please?
> >>
> >> At least we have some air to breath now again on jenkins.
> >>
> >> Olaf
> >>
> >>
> >>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
> >>>
> >>> Regarding setup of jenkins, you can find a lot of info here:
> >>>
> >>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
> >>>
> >>> Olaf
> >>>
> >>>
> >>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
> >>>>
> >>>> I am working with luca to get him access ..
> >>>>
> >>>> Olaf
> >>>>
> >>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>
> >>>>> Keep reporting issues, sorry :)
> >>>>>
> >>>>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
> >>>>> disk space issues on the Jenkins node, builds are stopped for the
> >>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I
> >>>>> guess it means accessing to the VM/hosts that are running Jenkins and
> >>>>> I have never done it (not even sure where to put my ssh key etc, what
> >>>>> is the hostname, etc..).
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>> Luca
> >>>>>
> >>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>>>
> >>>>>> Hi Luca,
> >>>>>>
> >>>>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
> >>>>>>
> >>>>>> Olaf
> >>>>>>
> >>>>>>
> >>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>>>
> >>>>>>> Hi Olaf,
> >>>>>>>
> >>>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
> >>>>>>> is valid though (http redirects to https), but I don't get why we
> >>>>>>> should delete the line (also the code returns a HTTP 201 afaics so it
> >>>>>>> seems not failing to find it).
> >>>>>>>
> >>>>>>> Lemme know, I am probably missing something, too many new things :)
> >>>>>>>
> >>>>>>> Luca
> >>>>>>>
> >>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>>>>>
> >>>>>>>> Hi Luca,
> >>>>>>>>
> >>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
> >>>>>>>>
> >>>>>>>> Try to delete this line:
> >>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
> >>>>>>>>
> >>>>>>>> Olaf
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>>>>>
> >>>>>>>>> Hi Olaf,
> >>>>>>>>>
> >>>>>>>>> I restarted the jobs and some packages started to build, thanks! I
> >>>>>>>>> found another weird error though:
> >>>>>>>>>
> >>>>>>>>> """
> >>>>>>>>> * What went wrong:
> >>>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
> >>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
> >>>>>>>>> """
> >>>>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
> >>>>>>>>>
> >>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
> >>>>>>>>>
> >>>>>>>>> Luca
> >>>>>>>>>
> >>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi Luca,
> >>>>>>>>>>
> >>>>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> >>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
> >>>>>>>>>>
> >>>>>>>>>> If this network is not present you might get this kind of error
> >>>>>>>>>> So try to restart nexus.
> >>>>>>>>>>
> >>>>>>>>>> Best
> >>>>>>>>>> Olaf
> >>>>>>>>>>
> >>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>>>>>>>
> >>>>>>>>>>> Update:
> >>>>>>>>>>>
> >>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
> >>>>>>>>>>> periodic execution.
> >>>>>>>>>>>
> >>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
> >>>>>>>>>>> builds were failing for the same error:
> >>>>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> >>>>>>>>>>> docker: Error response from daemon: cannot join network of a non
> >>>>>>>>>>> running container
> >>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
> >>>>>>>>>>>
> >>>>>>>>>>> I have probably missed something, I don't exactly understand the error
> >>>>>>>>>>> so any help would be appreciated :)
> >>>>>>>>>>>
> >>>>>>>>>>> Luca
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> >>>>>>>>>>>> package build. I can also enable periodic execution but I am not sure
> >>>>>>>>>>>> what I should put in the related time interval field (I didn't find it
> >>>>>>>>>>>> in the docs).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Luca
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Olaf! Thanks a lot :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
> >>>>>>>>>>>>> Ubuntu that seems unrelated, see
> >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
> >>>>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
> >>>>>>>>>>>>> 11 images published to Dockerhub.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> IIUC the last step is to modify
> >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
> >>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> >>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
> >>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Luca
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Luca,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I gave you super powers.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best
> >>>>>>>>>>>>>> Olaf
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi everybody!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> >>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> >>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> >>>>>>>>>>>>>>> the process and help out :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Luca
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi Luca,

the name changed. There is now some space available.

Check with the EC2 console to see the current public IP adresses .
https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running

Best
  Olaf

> Am 04.12.2021 um 18:23 schrieb Luca Toscano <to...@gmail.com>:
> 
> Hi :)
> 
> https://ci.bigtop.apache.org/log/all seems to indicate that we are
> again in trouble, docker-slave-02 seems full. I got the credentials to
> access the node but I can reach it (my ssh session hangs
> indefinitely), is there anybody that can take a look?
> 
> Thanks,
> 
> Luca
> 
> On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>> 
>> Hi,
>> 
>> I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
>> I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk.
>> I removed all bug the last ~3000 job dirs.
>> 
>> Can someone please look at the config of „bigpetstore“, please?
>> 
>> At least we have some air to breath now again on jenkins.
>> 
>> Olaf
>> 
>> 
>>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
>>> 
>>> Regarding setup of jenkins, you can find a lot of info here:
>>> 
>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
>>> 
>>> Olaf
>>> 
>>> 
>>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
>>>> 
>>>> I am working with luca to get him access ..
>>>> 
>>>> Olaf
>>>> 
>>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>> 
>>>>> Keep reporting issues, sorry :)
>>>>> 
>>>>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
>>>>> disk space issues on the Jenkins node, builds are stopped for the
>>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I
>>>>> guess it means accessing to the VM/hosts that are running Jenkins and
>>>>> I have never done it (not even sure where to put my ssh key etc, what
>>>>> is the hostname, etc..).
>>>>> 
>>>>> Thanks!
>>>>> 
>>>>> Luca
>>>>> 
>>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>>>> 
>>>>>> Hi Luca,
>>>>>> 
>>>>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
>>>>>> 
>>>>>> Olaf
>>>>>> 
>>>>>> 
>>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>>>> 
>>>>>>> Hi Olaf,
>>>>>>> 
>>>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
>>>>>>> is valid though (http redirects to https), but I don't get why we
>>>>>>> should delete the line (also the code returns a HTTP 201 afaics so it
>>>>>>> seems not failing to find it).
>>>>>>> 
>>>>>>> Lemme know, I am probably missing something, too many new things :)
>>>>>>> 
>>>>>>> Luca
>>>>>>> 
>>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>>>>>> 
>>>>>>>> Hi Luca,
>>>>>>>> 
>>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
>>>>>>>> 
>>>>>>>> Try to delete this line:
>>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
>>>>>>>> 
>>>>>>>> Olaf
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>>>>>> 
>>>>>>>>> Hi Olaf,
>>>>>>>>> 
>>>>>>>>> I restarted the jobs and some packages started to build, thanks! I
>>>>>>>>> found another weird error though:
>>>>>>>>> 
>>>>>>>>> """
>>>>>>>>> * What went wrong:
>>>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
>>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
>>>>>>>>> """
>>>>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
>>>>>>>>> 
>>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
>>>>>>>>> 
>>>>>>>>> Luca
>>>>>>>>> 
>>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Luca,
>>>>>>>>>> 
>>>>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
>>>>>>>>>> 
>>>>>>>>>> If this network is not present you might get this kind of error
>>>>>>>>>> So try to restart nexus.
>>>>>>>>>> 
>>>>>>>>>> Best
>>>>>>>>>> Olaf
>>>>>>>>>> 
>>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>>>>>>>> 
>>>>>>>>>>> Update:
>>>>>>>>>>> 
>>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>>>>>>>>>> periodic execution.
>>>>>>>>>>> 
>>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
>>>>>>>>>>> builds were failing for the same error:
>>>>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>>>>>>>>>> docker: Error response from daemon: cannot join network of a non
>>>>>>>>>>> running container
>>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
>>>>>>>>>>> 
>>>>>>>>>>> I have probably missed something, I don't exactly understand the error
>>>>>>>>>>> so any help would be appreciated :)
>>>>>>>>>>> 
>>>>>>>>>>> Luca
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>>>>>>>>>> package build. I can also enable periodic execution but I am not sure
>>>>>>>>>>>> what I should put in the related time interval field (I didn't find it
>>>>>>>>>>>> in the docs).
>>>>>>>>>>>> 
>>>>>>>>>>>> Luca
>>>>>>>>>>>> 
>>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Olaf! Thanks a lot :)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>>>>>>>>>> Ubuntu that seems unrelated, see
>>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
>>>>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>>>>>>>>>> 11 images published to Dockerhub.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> IIUC the last step is to modify
>>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
>>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Luca
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi Luca,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I gave you super powers.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Best
>>>>>>>>>>>>>> Olaf
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi everybody!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>>>>>>>>>> the process and help out :)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Luca
>>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>>

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Hi :)

https://ci.bigtop.apache.org/log/all seems to indicate that we are
again in trouble, docker-slave-02 seems full. I got the credentials to
access the node but I can reach it (my ssh session hangs
indefinitely), is there anybody that can take a look?

Thanks,

Luca

On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi,
>
> I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
> I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk.
> I removed all bug the last ~3000 job dirs.
>
> Can someone please look at the config of „bigpetstore“, please?
>
> At least we have some air to breath now again on jenkins.
>
> Olaf
>
>
> > Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
> >
> > Regarding setup of jenkins, you can find a lot of info here:
> >
> > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
> >
> > Olaf
> >
> >
> >> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
> >>
> >> I am working with luca to get him access ..
> >>
> >> Olaf
> >>
> >>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>
> >>> Keep reporting issues, sorry :)
> >>>
> >>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
> >>> disk space issues on the Jenkins node, builds are stopped for the
> >>> moment. No idea if there is a guide/tutorial for the cleanup, but I
> >>> guess it means accessing to the VM/hosts that are running Jenkins and
> >>> I have never done it (not even sure where to put my ssh key etc, what
> >>> is the hostname, etc..).
> >>>
> >>> Thanks!
> >>>
> >>> Luca
> >>>
> >>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>
> >>>> Hi Luca,
> >>>>
> >>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
> >>>>
> >>>> Olaf
> >>>>
> >>>>
> >>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>
> >>>>> Hi Olaf,
> >>>>>
> >>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
> >>>>> is valid though (http redirects to https), but I don't get why we
> >>>>> should delete the line (also the code returns a HTTP 201 afaics so it
> >>>>> seems not failing to find it).
> >>>>>
> >>>>> Lemme know, I am probably missing something, too many new things :)
> >>>>>
> >>>>> Luca
> >>>>>
> >>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>>>
> >>>>>> Hi Luca,
> >>>>>>
> >>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
> >>>>>>
> >>>>>> Try to delete this line:
> >>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
> >>>>>>
> >>>>>> Olaf
> >>>>>>
> >>>>>>
> >>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>>>
> >>>>>>> Hi Olaf,
> >>>>>>>
> >>>>>>> I restarted the jobs and some packages started to build, thanks! I
> >>>>>>> found another weird error though:
> >>>>>>>
> >>>>>>> """
> >>>>>>> * What went wrong:
> >>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
> >>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
> >>>>>>> """
> >>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
> >>>>>>>
> >>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
> >>>>>>>
> >>>>>>> Luca
> >>>>>>>
> >>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
> >>>>>>>>
> >>>>>>>> Hi Luca,
> >>>>>>>>
> >>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> >>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
> >>>>>>>>
> >>>>>>>> If this network is not present you might get this kind of error
> >>>>>>>> So try to restart nexus.
> >>>>>>>>
> >>>>>>>> Best
> >>>>>>>> Olaf
> >>>>>>>>
> >>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
> >>>>>>>>>
> >>>>>>>>> Update:
> >>>>>>>>>
> >>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
> >>>>>>>>> periodic execution.
> >>>>>>>>>
> >>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
> >>>>>>>>> builds were failing for the same error:
> >>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> >>>>>>>>> docker: Error response from daemon: cannot join network of a non
> >>>>>>>>> running container
> >>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
> >>>>>>>>>
> >>>>>>>>> I have probably missed something, I don't exactly understand the error
> >>>>>>>>> so any help would be appreciated :)
> >>>>>>>>>
> >>>>>>>>> Luca
> >>>>>>>>>
> >>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> >>>>>>>>>> package build. I can also enable periodic execution but I am not sure
> >>>>>>>>>> what I should put in the related time interval field (I didn't find it
> >>>>>>>>>> in the docs).
> >>>>>>>>>>
> >>>>>>>>>> Luca
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Olaf! Thanks a lot :)
> >>>>>>>>>>>
> >>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
> >>>>>>>>>>> Ubuntu that seems unrelated, see
> >>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
> >>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
> >>>>>>>>>>> 11 images published to Dockerhub.
> >>>>>>>>>>>
> >>>>>>>>>>> IIUC the last step is to modify
> >>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
> >>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> >>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
> >>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
> >>>>>>>>>>>
> >>>>>>>>>>> Luca
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Luca,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I gave you super powers.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best
> >>>>>>>>>>>> Olaf
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi everybody!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> >>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> >>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> >>>>>>>>>>>>> the process and help out :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Luca
> >>>>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> >
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi,

I had a quick look around. Job History was excessive: availlable back to 2017, I deleted all until 2019 and set the maximum life time to one year.
I see that not any job from bigtopstore have been removed so far. Now having 52995 jobs on disk. 
I removed all bug the last ~3000 job dirs. 

Can someone please look at the config of „bigpetstore“, please?

At least we have some air to breath now again on jenkins.

Olaf


> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <of...@oflebbe.de>:
> 
> Regarding setup of jenkins, you can find a lot of info here:
> 
> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>
> 
> Olaf
> 
> 
>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>>:
>> 
>> I am working with luca to get him access ..
>> 
>> Olaf
>> 
>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>> 
>>> Keep reporting issues, sorry :)
>>> 
>>> I see from https://ci.bigtop.apache.org/log/all <https://ci.bigtop.apache.org/log/all> that we are having
>>> disk space issues on the Jenkins node, builds are stopped for the
>>> moment. No idea if there is a guide/tutorial for the cleanup, but I
>>> guess it means accessing to the VM/hosts that are running Jenkins and
>>> I have never done it (not even sure where to put my ssh key etc, what
>>> is the hostname, etc..).
>>> 
>>> Thanks!
>>> 
>>> Luca
>>> 
>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>> 
>>>> Hi Luca,
>>>> 
>>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
>>>> 
>>>> Olaf
>>>> 
>>>> 
>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>> 
>>>>> Hi Olaf,
>>>>> 
>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ <https://repository.jboss.org/nexus/content/groups/public/>
>>>>> is valid though (http redirects to https), but I don't get why we
>>>>> should delete the line (also the code returns a HTTP 201 afaics so it
>>>>> seems not failing to find it).
>>>>> 
>>>>> Lemme know, I am probably missing something, too many new things :)
>>>>> 
>>>>> Luca
>>>>> 
>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>>>> 
>>>>>> Hi Luca,
>>>>>> 
>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ <http://jboss.org/>> repository does not exist any more.
>>>>>> 
>>>>>> Try to delete this line:
>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>>
>>>>>> 
>>>>>> Olaf
>>>>>> 
>>>>>> 
>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>>>> 
>>>>>>> Hi Olaf,
>>>>>>> 
>>>>>>> I restarted the jobs and some packages started to build, thanks! I
>>>>>>> found another weird error though:
>>>>>>> 
>>>>>>> """
>>>>>>> * What went wrong:
>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org <http://configure-nexus-repository.jboss.org/>'.
>>>>>>>> Failed to configure Nexus proxy repository.jboss.org <http://repository.jboss.org/> with http code 201 returned. Run with --info option to see the executed command
>>>>>>> """
>>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console>
>>>>>>> 
>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 <https://github.com/apache/bigtop/pull/834> could help :)
>>>>>>> 
>>>>>>> Luca
>>>>>>> 
>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of@oflebbe.de <ma...@oflebbe.de>> wrote:
>>>>>>>> 
>>>>>>>> Hi Luca,
>>>>>>>> 
>>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/> <https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>>
>>>>>>>> 
>>>>>>>> If this network is not present you might get this kind of error
>>>>>>>> So try to restart nexus.
>>>>>>>> 
>>>>>>>> Best
>>>>>>>> Olaf
>>>>>>>> 
>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>>:
>>>>>>>>> 
>>>>>>>>> Update:
>>>>>>>>> 
>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>>>>>>>> periodic execution.
>>>>>>>>> 
>>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
>>>>>>>>> builds were failing for the same error:
>>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>>>>>>>> docker: Error response from daemon: cannot join network of a non
>>>>>>>>> running container
>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console>
>>>>>>>>> 
>>>>>>>>> I have probably missed something, I don't exactly understand the error
>>>>>>>>> so any help would be appreciated :)
>>>>>>>>> 
>>>>>>>>> Luca
>>>>>>>>> 
>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>>>>>>>> package build. I can also enable periodic execution but I am not sure
>>>>>>>>>> what I should put in the related time interval field (I didn't find it
>>>>>>>>>> in the docs).
>>>>>>>>>> 
>>>>>>>>>> Luca
>>>>>>>>>> 
>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <toscano.luca@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Olaf! Thanks a lot :)
>>>>>>>>>>> 
>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>>>>>>>> Ubuntu that seems unrelated, see
>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>)
>>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>>>>>>>> 11 images published to Dockerhub.
>>>>>>>>>>> 
>>>>>>>>>>> IIUC the last step is to modify
>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/>
>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>>>>>>>> 
>>>>>>>>>>> Luca
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Luca,
>>>>>>>>>>>> 
>>>>>>>>>>>> I gave you super powers.
>>>>>>>>>>>> 
>>>>>>>>>>>> Best
>>>>>>>>>>>> Olaf
>>>>>>>>>>>> 
>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi everybody!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>>>>>>>> the process and help out :)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Luca
>>>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>> 
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Regarding setup of jenkins, you can find a lot of info here:

https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide>

Olaf


> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <of...@oflebbe.de>:
> 
> I am working with luca to get him access ..
> 
> Olaf
> 
>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <to...@gmail.com>:
>> 
>> Keep reporting issues, sorry :)
>> 
>> I see from https://ci.bigtop.apache.org/log/all that we are having
>> disk space issues on the Jenkins node, builds are stopped for the
>> moment. No idea if there is a guide/tutorial for the cleanup, but I
>> guess it means accessing to the VM/hosts that are running Jenkins and
>> I have never done it (not even sure where to put my ssh key etc, what
>> is the hostname, etc..).
>> 
>> Thanks!
>> 
>> Luca
>> 
>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>> 
>>> Hi Luca,
>>> 
>>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
>>> 
>>> Olaf
>>> 
>>> 
>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <to...@gmail.com>:
>>>> 
>>>> Hi Olaf,
>>>> 
>>>> I see that https://repository.jboss.org/nexus/content/groups/public/
>>>> is valid though (http redirects to https), but I don't get why we
>>>> should delete the line (also the code returns a HTTP 201 afaics so it
>>>> seems not failing to find it).
>>>> 
>>>> Lemme know, I am probably missing something, too many new things :)
>>>> 
>>>> Luca
>>>> 
>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>> 
>>>>> Hi Luca,
>>>>> 
>>>>> Seems like jboss.org <http://jboss.org/> repository does not exist any more.
>>>>> 
>>>>> Try to delete this line:
>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>
>>>>> 
>>>>> Olaf
>>>>> 
>>>>> 
>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
>>>>>> 
>>>>>> Hi Olaf,
>>>>>> 
>>>>>> I restarted the jobs and some packages started to build, thanks! I
>>>>>> found another weird error though:
>>>>>> 
>>>>>> """
>>>>>> * What went wrong:
>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org'.
>>>>>>> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
>>>>>> """
>>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
>>>>>> 
>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
>>>>>> 
>>>>>> Luca
>>>>>> 
>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>> 
>>>>>>> Hi Luca,
>>>>>>> 
>>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
>>>>>>> 
>>>>>>> If this network is not present you might get this kind of error
>>>>>>> So try to restart nexus.
>>>>>>> 
>>>>>>> Best
>>>>>>> Olaf
>>>>>>> 
>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>> 
>>>>>>>> Update:
>>>>>>>> 
>>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>>>>>>> periodic execution.
>>>>>>>> 
>>>>>>>> - Tried to kick off a build, but I stopped it since all the package
>>>>>>>> builds were failing for the same error:
>>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>>>>>>> docker: Error response from daemon: cannot join network of a non
>>>>>>>> running container
>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
>>>>>>>> 
>>>>>>>> I have probably missed something, I don't exactly understand the error
>>>>>>>> so any help would be appreciated :)
>>>>>>>> 
>>>>>>>> Luca
>>>>>>>> 
>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>>>>>>> package build. I can also enable periodic execution but I am not sure
>>>>>>>>> what I should put in the related time interval field (I didn't find it
>>>>>>>>> in the docs).
>>>>>>>>> 
>>>>>>>>> Luca
>>>>>>>>> 
>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Olaf! Thanks a lot :)
>>>>>>>>>> 
>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>>>>>>> Ubuntu that seems unrelated, see
>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
>>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>>>>>>> 11 images published to Dockerhub.
>>>>>>>>>> 
>>>>>>>>>> IIUC the last step is to modify
>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>>>>>>> 
>>>>>>>>>> Luca
>>>>>>>>>> 
>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Luca,
>>>>>>>>>>> 
>>>>>>>>>>> I gave you super powers.
>>>>>>>>>>> 
>>>>>>>>>>> Best
>>>>>>>>>>> Olaf
>>>>>>>>>>> 
>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi everybody!
>>>>>>>>>>>> 
>>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>>>>>>> the process and help out :)
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> 
>>>>>>>>>>>> Luca
>>>>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

I am working with luca to get him access ..

Olaf

> Am 27.11.2021 um 18:29 schrieb Luca Toscano <to...@gmail.com>:
> 
> Keep reporting issues, sorry :)
> 
> I see from https://ci.bigtop.apache.org/log/all that we are having
> disk space issues on the Jenkins node, builds are stopped for the
> moment. No idea if there is a guide/tutorial for the cleanup, but I
> guess it means accessing to the VM/hosts that are running Jenkins and
> I have never done it (not even sure where to put my ssh key etc, what
> is the hostname, etc..).
> 
> Thanks!
> 
> Luca
> 
> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>> 
>> Hi Luca,
>> 
>> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
>> 
>> Olaf
>> 
>> 
>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <to...@gmail.com>:
>>> 
>>> Hi Olaf,
>>> 
>>> I see that https://repository.jboss.org/nexus/content/groups/public/
>>> is valid though (http redirects to https), but I don't get why we
>>> should delete the line (also the code returns a HTTP 201 afaics so it
>>> seems not failing to find it).
>>> 
>>> Lemme know, I am probably missing something, too many new things :)
>>> 
>>> Luca
>>> 
>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>> 
>>>> Hi Luca,
>>>> 
>>>> Seems like jboss.org <http://jboss.org/> repository does not exist any more.
>>>> 
>>>> Try to delete this line:
>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>
>>>> 
>>>> Olaf
>>>> 
>>>> 
>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
>>>>> 
>>>>> Hi Olaf,
>>>>> 
>>>>> I restarted the jobs and some packages started to build, thanks! I
>>>>> found another weird error though:
>>>>> 
>>>>> """
>>>>> * What went wrong:
>>>>> Execution failed for task ':configure-nexus-repository.jboss.org'.
>>>>>> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
>>>>> """
>>>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
>>>>> 
>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
>>>>> 
>>>>> Luca
>>>>> 
>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>> 
>>>>>> Hi Luca,
>>>>>> 
>>>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
>>>>>> 
>>>>>> If this network is not present you might get this kind of error
>>>>>> So try to restart nexus.
>>>>>> 
>>>>>> Best
>>>>>> Olaf
>>>>>> 
>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>> 
>>>>>>> Update:
>>>>>>> 
>>>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>>>>>> periodic execution.
>>>>>>> 
>>>>>>> - Tried to kick off a build, but I stopped it since all the package
>>>>>>> builds were failing for the same error:
>>>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>>>>>> docker: Error response from daemon: cannot join network of a non
>>>>>>> running container
>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
>>>>>>> 
>>>>>>> I have probably missed something, I don't exactly understand the error
>>>>>>> so any help would be appreciated :)
>>>>>>> 
>>>>>>> Luca
>>>>>>> 
>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>>>>>> package build. I can also enable periodic execution but I am not sure
>>>>>>>> what I should put in the related time interval field (I didn't find it
>>>>>>>> in the docs).
>>>>>>>> 
>>>>>>>> Luca
>>>>>>>> 
>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Olaf! Thanks a lot :)
>>>>>>>>> 
>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>>>>>> Ubuntu that seems unrelated, see
>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
>>>>>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>>>>>> 11 images published to Dockerhub.
>>>>>>>>> 
>>>>>>>>> IIUC the last step is to modify
>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>>>>>> 
>>>>>>>>> Luca
>>>>>>>>> 
>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Luca,
>>>>>>>>>> 
>>>>>>>>>> I gave you super powers.
>>>>>>>>>> 
>>>>>>>>>> Best
>>>>>>>>>> Olaf
>>>>>>>>>> 
>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>>>>> 
>>>>>>>>>>> Hi everybody!
>>>>>>>>>>> 
>>>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>>>>>> the process and help out :)
>>>>>>>>>>> 
>>>>>>>>>>> Thanks!
>>>>>>>>>>> 
>>>>>>>>>>> Luca
>>>>>>>>>> 
>>>>>> 
>>>> 
>>

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Keep reporting issues, sorry :)

I see from https://ci.bigtop.apache.org/log/all that we are having
disk space issues on the Jenkins node, builds are stopped for the
moment. No idea if there is a guide/tutorial for the cleanup, but I
guess it means accessing to the VM/hosts that are running Jenkins and
I have never done it (not even sure where to put my ssh key etc, what
is the hostname, etc..).

Thanks!

Luca

On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Luca,
>
> you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.
>
> Olaf
>
>
> > Am 26.11.2021 um 19:06 schrieb Luca Toscano <to...@gmail.com>:
> >
> > Hi Olaf,
> >
> > I see that https://repository.jboss.org/nexus/content/groups/public/
> > is valid though (http redirects to https), but I don't get why we
> > should delete the line (also the code returns a HTTP 201 afaics so it
> > seems not failing to find it).
> >
> > Lemme know, I am probably missing something, too many new things :)
> >
> > Luca
> >
> > On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>
> >> Hi Luca,
> >>
> >> Seems like jboss.org <http://jboss.org/> repository does not exist any more.
> >>
> >> Try to delete this line:
> >> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>
> >>
> >> Olaf
> >>
> >>
> >>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
> >>>
> >>> Hi Olaf,
> >>>
> >>> I restarted the jobs and some packages started to build, thanks! I
> >>> found another weird error though:
> >>>
> >>> """
> >>> * What went wrong:
> >>> Execution failed for task ':configure-nexus-repository.jboss.org'.
> >>>> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
> >>> """
> >>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
> >>>
> >>> I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
> >>>
> >>> Luca
> >>>
> >>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>
> >>>> Hi Luca,
> >>>>
> >>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> >>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
> >>>>
> >>>> If this network is not present you might get this kind of error
> >>>> So try to restart nexus.
> >>>>
> >>>> Best
> >>>> Olaf
> >>>>
> >>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>
> >>>>> Update:
> >>>>>
> >>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
> >>>>> periodic execution.
> >>>>>
> >>>>> - Tried to kick off a build, but I stopped it since all the package
> >>>>> builds were failing for the same error:
> >>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> >>>>> docker: Error response from daemon: cannot join network of a non
> >>>>> running container
> >>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
> >>>>>
> >>>>> I have probably missed something, I don't exactly understand the error
> >>>>> so any help would be appreciated :)
> >>>>>
> >>>>> Luca
> >>>>>
> >>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
> >>>>>>
> >>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> >>>>>> package build. I can also enable periodic execution but I am not sure
> >>>>>> what I should put in the related time interval field (I didn't find it
> >>>>>> in the docs).
> >>>>>>
> >>>>>> Luca
> >>>>>>
> >>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Hi Olaf! Thanks a lot :)
> >>>>>>>
> >>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
> >>>>>>> Ubuntu that seems unrelated, see
> >>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
> >>>>>>> and I was able to build the hadoop packages locally via the new Debian
> >>>>>>> 11 images published to Dockerhub.
> >>>>>>>
> >>>>>>> IIUC the last step is to modify
> >>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> >>>>>>> periodic execution for preparing 3.0.0 release." so before doing
> >>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
> >>>>>>>
> >>>>>>> Luca
> >>>>>>>
> >>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>>>>>
> >>>>>>>> Hi Luca,
> >>>>>>>>
> >>>>>>>> I gave you super powers.
> >>>>>>>>
> >>>>>>>> Best
> >>>>>>>> Olaf
> >>>>>>>>
> >>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>> Hi everybody!
> >>>>>>>>>
> >>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> >>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> >>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> >>>>>>>>> the process and help out :)
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>> Luca
> >>>>>>>>
> >>>>
> >>
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi Luca,

you are right, 201 should be valid. However there seems to be some compatibility problem with at least this repo. So comment it out for now if it resolves the isse. Or remove the whole proxy stuff altoghether. We had stability issues with the internet connectivity at that time. The proxy resolved that mostly.

Olaf


> Am 26.11.2021 um 19:06 schrieb Luca Toscano <to...@gmail.com>:
> 
> Hi Olaf,
> 
> I see that https://repository.jboss.org/nexus/content/groups/public/
> is valid though (http redirects to https), but I don't get why we
> should delete the line (also the code returns a HTTP 201 afaics so it
> seems not failing to find it).
> 
> Lemme know, I am probably missing something, too many new things :)
> 
> Luca
> 
> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>> 
>> Hi Luca,
>> 
>> Seems like jboss.org <http://jboss.org/> repository does not exist any more.
>> 
>> Try to delete this line:
>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>
>> 
>> Olaf
>> 
>> 
>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
>>> 
>>> Hi Olaf,
>>> 
>>> I restarted the jobs and some packages started to build, thanks! I
>>> found another weird error though:
>>> 
>>> """
>>> * What went wrong:
>>> Execution failed for task ':configure-nexus-repository.jboss.org'.
>>>> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
>>> """
>>> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
>>> 
>>> I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
>>> 
>>> Luca
>>> 
>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>> 
>>>> Hi Luca,
>>>> 
>>>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>>>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
>>>> 
>>>> If this network is not present you might get this kind of error
>>>> So try to restart nexus.
>>>> 
>>>> Best
>>>> Olaf
>>>> 
>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
>>>>> 
>>>>> Update:
>>>>> 
>>>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>>>> periodic execution.
>>>>> 
>>>>> - Tried to kick off a build, but I stopped it since all the package
>>>>> builds were failing for the same error:
>>>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>>>> docker: Error response from daemon: cannot join network of a non
>>>>> running container
>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
>>>>> 
>>>>> I have probably missed something, I don't exactly understand the error
>>>>> so any help would be appreciated :)
>>>>> 
>>>>> Luca
>>>>> 
>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>> 
>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>>>> package build. I can also enable periodic execution but I am not sure
>>>>>> what I should put in the related time interval field (I didn't find it
>>>>>> in the docs).
>>>>>> 
>>>>>> Luca
>>>>>> 
>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>>>>>>> 
>>>>>>> Hi Olaf! Thanks a lot :)
>>>>>>> 
>>>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>>>> Ubuntu that seems unrelated, see
>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
>>>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>>>> 11 images published to Dockerhub.
>>>>>>> 
>>>>>>> IIUC the last step is to modify
>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>>>> 
>>>>>>> Luca
>>>>>>> 
>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>>>> 
>>>>>>>> Hi Luca,
>>>>>>>> 
>>>>>>>> I gave you super powers.
>>>>>>>> 
>>>>>>>> Best
>>>>>>>> Olaf
>>>>>>>> 
>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>>>> 
>>>>>>>>> Hi everybody!
>>>>>>>>> 
>>>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>>>> the process and help out :)
>>>>>>>>> 
>>>>>>>>> Thanks!
>>>>>>>>> 
>>>>>>>>> Luca
>>>>>>>> 
>>>> 
>>

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Hi Olaf,

I see that https://repository.jboss.org/nexus/content/groups/public/
is valid though (http redirects to https), but I don't get why we
should delete the line (also the code returns a HTTP 201 afaics so it
seems not failing to find it).

Lemme know, I am probably missing something, too many new things :)

Luca

On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Luca,
>
> Seems like jboss.org <http://jboss.org/> repository does not exist any more.
>
> Try to delete this line:
> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>
>
> Olaf
>
>
> > Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
> >
> > Hi Olaf,
> >
> > I restarted the jobs and some packages started to build, thanks! I
> > found another weird error though:
> >
> > """
> > * What went wrong:
> > Execution failed for task ':configure-nexus-repository.jboss.org'.
> >> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
> > """
> > More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
> >
> > I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
> >
> > Luca
> >
> > On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>
> >> Hi Luca,
> >>
> >> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> >> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
> >>
> >> If this network is not present you might get this kind of error
> >> So try to restart nexus.
> >>
> >> Best
> >> Olaf
> >>
> >>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
> >>>
> >>> Update:
> >>>
> >>> - Found the config diff changes in Jenkins, so I know how to re-enable
> >>> periodic execution.
> >>>
> >>> - Tried to kick off a build, but I stopped it since all the package
> >>> builds were failing for the same error:
> >>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> >>> docker: Error response from daemon: cannot join network of a non
> >>> running container
> >>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
> >>>
> >>> I have probably missed something, I don't exactly understand the error
> >>> so any help would be appreciated :)
> >>>
> >>> Luca
> >>>
> >>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
> >>>>
> >>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> >>>> package build. I can also enable periodic execution but I am not sure
> >>>> what I should put in the related time interval field (I didn't find it
> >>>> in the docs).
> >>>>
> >>>> Luca
> >>>>
> >>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
> >>>>>
> >>>>> Hi Olaf! Thanks a lot :)
> >>>>>
> >>>>> I managed to change CI to allow Debian 11 (minus a little error for
> >>>>> Ubuntu that seems unrelated, see
> >>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
> >>>>> and I was able to build the hadoop packages locally via the new Debian
> >>>>> 11 images published to Dockerhub.
> >>>>>
> >>>>> IIUC the last step is to modify
> >>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>>>> and kick off a build, but I see "2021-10-02: temporarily disable
> >>>>> periodic execution for preparing 3.0.0 release." so before doing
> >>>>> anything I'd like some feedback (to avoid causing problems etc..).
> >>>>>
> >>>>> Luca
> >>>>>
> >>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>>>
> >>>>>> Hi Luca,
> >>>>>>
> >>>>>> I gave you super powers.
> >>>>>>
> >>>>>> Best
> >>>>>> Olaf
> >>>>>>
> >>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>>>
> >>>>>>> Hi everybody!
> >>>>>>>
> >>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> >>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> >>>>>>> permissions. If anybody has some time to help I'd be happy to learn
> >>>>>>> the process and help out :)
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>> Luca
> >>>>>>
> >>
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi Luca,

Seems like jboss.org <http://jboss.org/> repository does not exist any more.

Try to delete this line:
https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>

Olaf


> Am 26.11.2021 um 14:23 schrieb Luca Toscano <to...@gmail.com>:
> 
> Hi Olaf,
> 
> I restarted the jobs and some packages started to build, thanks! I
> found another weird error though:
> 
> """
> * What went wrong:
> Execution failed for task ':configure-nexus-repository.jboss.org'.
>> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
> """
> More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console
> 
> I am wondering if https://github.com/apache/bigtop/pull/834 could help :)
> 
> Luca
> 
> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>> 
>> Hi Luca,
>> 
>> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
>> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
>> 
>> If this network is not present you might get this kind of error
>> So try to restart nexus.
>> 
>> Best
>> Olaf
>> 
>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
>>> 
>>> Update:
>>> 
>>> - Found the config diff changes in Jenkins, so I know how to re-enable
>>> periodic execution.
>>> 
>>> - Tried to kick off a build, but I stopped it since all the package
>>> builds were failing for the same error:
>>> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
>>> docker: Error response from daemon: cannot join network of a non
>>> running container
>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
>>> 
>>> I have probably missed something, I don't exactly understand the error
>>> so any help would be appreciated :)
>>> 
>>> Luca
>>> 
>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>>>> 
>>>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>>>> package build. I can also enable periodic execution but I am not sure
>>>> what I should put in the related time interval field (I didn't find it
>>>> in the docs).
>>>> 
>>>> Luca
>>>> 
>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>>>>> 
>>>>> Hi Olaf! Thanks a lot :)
>>>>> 
>>>>> I managed to change CI to allow Debian 11 (minus a little error for
>>>>> Ubuntu that seems unrelated, see
>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
>>>>> and I was able to build the hadoop packages locally via the new Debian
>>>>> 11 images published to Dockerhub.
>>>>> 
>>>>> IIUC the last step is to modify
>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>>>> periodic execution for preparing 3.0.0 release." so before doing
>>>>> anything I'd like some feedback (to avoid causing problems etc..).
>>>>> 
>>>>> Luca
>>>>> 
>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>>>> 
>>>>>> Hi Luca,
>>>>>> 
>>>>>> I gave you super powers.
>>>>>> 
>>>>>> Best
>>>>>> Olaf
>>>>>> 
>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>>>> 
>>>>>>> Hi everybody!
>>>>>>> 
>>>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>>>> the process and help out :)
>>>>>>> 
>>>>>>> Thanks!
>>>>>>> 
>>>>>>> Luca
>>>>>> 
>>

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Hi Olaf,

I restarted the jobs and some packages started to build, thanks! I
found another weird error though:

"""
* What went wrong:
Execution failed for task ':configure-nexus-repository.jboss.org'.
> Failed to configure Nexus proxy repository.jboss.org with http code 201 returned. Run with --info option to see the executed command
"""
More info: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console

I am wondering if https://github.com/apache/bigtop/pull/834 could help :)

Luca

On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Luca,
>
> I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
> https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>
>
> If this network is not present you might get this kind of error
> So try to restart nexus.
>
> Best
> Olaf
>
> > Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
> >
> > Update:
> >
> > - Found the config diff changes in Jenkins, so I know how to re-enable
> > periodic execution.
> >
> > - Tried to kick off a build, but I stopped it since all the package
> > builds were failing for the same error:
> > ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> > docker: Error response from daemon: cannot join network of a non
> > running container
> > Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
> >
> > I have probably missed something, I don't exactly understand the error
> > so any help would be appreciated :)
> >
> > Luca
> >
> > On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
> >>
> >> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> >> package build. I can also enable periodic execution but I am not sure
> >> what I should put in the related time interval field (I didn't find it
> >> in the docs).
> >>
> >> Luca
> >>
> >> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
> >>>
> >>> Hi Olaf! Thanks a lot :)
> >>>
> >>> I managed to change CI to allow Debian 11 (minus a little error for
> >>> Ubuntu that seems unrelated, see
> >>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
> >>> and I was able to build the hadoop packages locally via the new Debian
> >>> 11 images published to Dockerhub.
> >>>
> >>> IIUC the last step is to modify
> >>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>> and kick off a build, but I see "2021-10-02: temporarily disable
> >>> periodic execution for preparing 3.0.0 release." so before doing
> >>> anything I'd like some feedback (to avoid causing problems etc..).
> >>>
> >>> Luca
> >>>
> >>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>>>
> >>>> Hi Luca,
> >>>>
> >>>> I gave you super powers.
> >>>>
> >>>> Best
> >>>> Olaf
> >>>>
> >>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >>>>>
> >>>>> Hi everybody!
> >>>>>
> >>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> >>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
> >>>>> permissions. If anybody has some time to help I'd be happy to learn
> >>>>> the process and help out :)
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>> Luca
> >>>>
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi Luca,

I once added a nexus container as an download cache for maven dependencies, attached to a seperate docker network. It has to be started manually if it failed.
https://ci.bigtop.apache.org/job/nexus-restart/ <https://ci.bigtop.apache.org/job/nexus-restart/>

If this network is not present you might get this kind of error
So try to restart nexus.

Best
Olaf

> Am 26.11.2021 um 09:04 schrieb Luca Toscano <to...@gmail.com>:
> 
> Update:
> 
> - Found the config diff changes in Jenkins, so I know how to re-enable
> periodic execution.
> 
> - Tried to kick off a build, but I stopped it since all the package
> builds were failing for the same error:
> ++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
> docker: Error response from daemon: cannot join network of a non
> running container
> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console
> 
> I have probably missed something, I don't exactly understand the error
> so any help would be appreciated :)
> 
> Luca
> 
> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>> 
>> Update: I added debian-11 support, if nobody disagrees I'd kick off a
>> package build. I can also enable periodic execution but I am not sure
>> what I should put in the related time interval field (I didn't find it
>> in the docs).
>> 
>> Luca
>> 
>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>>> 
>>> Hi Olaf! Thanks a lot :)
>>> 
>>> I managed to change CI to allow Debian 11 (minus a little error for
>>> Ubuntu that seems unrelated, see
>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
>>> and I was able to build the hadoop packages locally via the new Debian
>>> 11 images published to Dockerhub.
>>> 
>>> IIUC the last step is to modify
>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>> and kick off a build, but I see "2021-10-02: temporarily disable
>>> periodic execution for preparing 3.0.0 release." so before doing
>>> anything I'd like some feedback (to avoid causing problems etc..).
>>> 
>>> Luca
>>> 
>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>>>> 
>>>> Hi Luca,
>>>> 
>>>> I gave you super powers.
>>>> 
>>>> Best
>>>> Olaf
>>>> 
>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
>>>>> 
>>>>> Hi everybody!
>>>>> 
>>>>> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
>>>>> IIUC I'd need to add the CI jobs now, but of course I don't have
>>>>> permissions. If anybody has some time to help I'd be happy to learn
>>>>> the process and help out :)
>>>>> 
>>>>> Thanks!
>>>>> 
>>>>> Luca
>>>>

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Update:

- Found the config diff changes in Jenkins, so I know how to re-enable
periodic execution.

- Tried to kick off a build, but I stopped it since all the package
builds were failing for the same error:
++ docker run -d --net=container:nexus bigtop/slaves:trunk-debian-10 /sbin/init
docker: Error response from daemon: cannot join network of a non
running container
Example: https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console

I have probably missed something, I don't exactly understand the error
so any help would be appreciated :)

Luca

On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano <to...@gmail.com> wrote:
>
> Update: I added debian-11 support, if nobody disagrees I'd kick off a
> package build. I can also enable periodic execution but I am not sure
> what I should put in the related time interval field (I didn't find it
> in the docs).
>
> Luca
>
> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
> >
> > Hi Olaf! Thanks a lot :)
> >
> > I managed to change CI to allow Debian 11 (minus a little error for
> > Ubuntu that seems unrelated, see
> > https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
> > and I was able to build the hadoop packages locally via the new Debian
> > 11 images published to Dockerhub.
> >
> > IIUC the last step is to modify
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > and kick off a build, but I see "2021-10-02: temporarily disable
> > periodic execution for preparing 3.0.0 release." so before doing
> > anything I'd like some feedback (to avoid causing problems etc..).
> >
> > Luca
> >
> > On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >
> > > Hi Luca,
> > >
> > > I gave you super powers.
> > >
> > > Best
> > > Olaf
> > >
> > > > Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> > > >
> > > > Hi everybody!
> > > >
> > > > I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> > > > IIUC I'd need to add the CI jobs now, but of course I don't have
> > > > permissions. If anybody has some time to help I'd be happy to learn
> > > > the process and help out :)
> > > >
> > > > Thanks!
> > > >
> > > > Luca
> > >

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Update: I added debian-11 support, if nobody disagrees I'd kick off a
package build. I can also enable periodic execution but I am not sure
what I should put in the related time interval field (I didn't find it
in the docs).

Luca

On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano <to...@gmail.com> wrote:
>
> Hi Olaf! Thanks a lot :)
>
> I managed to change CI to allow Debian 11 (minus a little error for
> Ubuntu that seems unrelated, see
> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
> and I was able to build the hadoop packages locally via the new Debian
> 11 images published to Dockerhub.
>
> IIUC the last step is to modify
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> and kick off a build, but I see "2021-10-02: temporarily disable
> periodic execution for preparing 3.0.0 release." so before doing
> anything I'd like some feedback (to avoid causing problems etc..).
>
> Luca
>
> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >
> > Hi Luca,
> >
> > I gave you super powers.
> >
> > Best
> > Olaf
> >
> > > Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> > >
> > > Hi everybody!
> > >
> > > I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> > > IIUC I'd need to add the CI jobs now, but of course I don't have
> > > permissions. If anybody has some time to help I'd be happy to learn
> > > the process and help out :)
> > >
> > > Thanks!
> > >
> > > Luca
> >

Re: Set up CI for Debian 11 Bullseye

Posted by Luca Toscano <to...@gmail.com>.

Hi Olaf! Thanks a lot :)

I managed to change CI to allow Debian 11 (minus a little error for
Ubuntu that seems unrelated, see
https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/)
and I was able to build the hadoop packages locally via the new Debian
11 images published to Dockerhub.

IIUC the last step is to modify
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
and kick off a build, but I see "2021-10-02: temporarily disable
periodic execution for preparing 3.0.0 release." so before doing
anything I'd like some feedback (to avoid causing problems etc..).

Luca

On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Luca,
>
> I gave you super powers.
>
> Best
> Olaf
>
> > Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> >
> > Hi everybody!
> >
> > I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> > IIUC I'd need to add the CI jobs now, but of course I don't have
> > permissions. If anybody has some time to help I'd be happy to learn
> > the process and help out :)
> >
> > Thanks!
> >
> > Luca
>

Re: Set up CI for Debian 11 Bullseye

Posted by Olaf Flebbe <of...@oflebbe.de>.

Hi Luca,

I gave you super powers.

Best
Olaf

> Am 19.11.2021 um 08:15 schrieb Luca Toscano <to...@gmail.com>:
> 
> Hi everybody!
> 
> I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and
> IIUC I'd need to add the CI jobs now, but of course I don't have
> permissions. If anybody has some time to help I'd be happy to learn
> the process and help out :)
> 
> Thanks!
> 
> Luca