You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Olaf Flebbe <of...@oflebbe.de> on 2016/01/08 18:43:28 UTC
Bigtop-trunk is all blue
Hi,
updating the CI Infrastructure made Bigtop-trunk all blue.
I will disable the nexus workaroung now for Bigtop-trunk-packages . A new bigtop/slaves:trunk-opensuse-13.2 image is pushed
Olaf
Re: Bigtop-trunk is all blue
Posted by Olaf Flebbe <of...@oflebbe.de>.
> Am 11.01.2016 um 22:59 schrieb Konstantin Boudnik <co...@apache.org>:
>
> On Mon, Jan 11, 2016 at 10:57PM, Olaf Flebbe wrote:
>>
>>> Am 11.01.2016 um 20:46 schrieb Konstantin Boudnik <co...@apache.org>:
>>>
>>> BTW, shall we be adding opensuse-13.2 to Bigtop-trunk job?
>>
>> Problem is opensuse uses a different repository mechanism "zypper" rather
>> redhat and debian derived distributions. AFAIK this hasn't been implemented.
>
> You're saying it hasn't been implemented for the toolchain or else? Cause we
> don't need package manager just to build the bigtop (well, except for the
> toolchain and deployment part).
>
Yep the toolchain. Bigtop-trunk creates repositories too, so this would be missing. Have a patch ready to use deployment directly from the CI job.
Since we cannot test without a repository, there is no usecase for running opensuse with Bigtop-trunk.
Olaf
Re: Bigtop-trunk is all blue
Posted by Konstantin Boudnik <co...@apache.org>.
On Mon, Jan 11, 2016 at 10:57PM, Olaf Flebbe wrote:
>
> > Am 11.01.2016 um 20:46 schrieb Konstantin Boudnik <co...@apache.org>:
> >
> > BTW, shall we be adding opensuse-13.2 to Bigtop-trunk job?
>
> Problem is opensuse uses a different repository mechanism "zypper" rather
> redhat and debian derived distributions. AFAIK this hasn't been implemented.
You're saying it hasn't been implemented for the toolchain or else? Cause we
don't need package manager just to build the bigtop (well, except for the
toolchain and deployment part).
Cos
Re: Bigtop-trunk is all blue
Posted by Olaf Flebbe <of...@oflebbe.de>.
> Am 11.01.2016 um 20:46 schrieb Konstantin Boudnik <co...@apache.org>:
>
> BTW, shall we be adding opensuse-13.2 to Bigtop-trunk job?
Problem is opensuse uses a different repository mechanism "zypper" rather redhat and debian derived distributions. AFAIK this hasn't been implemented.
Olaf
Re: Bigtop-trunk is all blue
Posted by Konstantin Boudnik <co...@apache.org>.
BTW, shall we be adding opensuse-13.2 to Bigtop-trunk job?
Cos
On Sat, Jan 09, 2016 at 09:22PM, Olaf Flebbe wrote:
> Hi,
>
>
>
> >
> > I think having a local mirroring Nexus server - either per the whole build, or
> > just standalone, might be very beneficial for the build time/failure rates.
> >
> > Cos
>
>
>
> I like to have it both the -trunk and -trunk-packages, too.
>
> Problem is how to start/stop it, since a build will fail almost immediatly if one restarts the nexus while a compile is running.
>
> One approach is to restart it manually. So I renamed the job to nexus-restart.
>
> If no job is running feel free to restart nexus (please wait ~10sec after this before starting jobs depending on it).
> The job uses the Node and Label parameter plugin to execute it on all docker-slaves.
>
> The build jobs are now changed to use the nexus containes. Hopefully they work now ;-)
>
> Olaf
>
> PS: The build failures are gone after updating our infra.
>
>
>
Re: Bigtop-trunk is all blue
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
>
> I think having a local mirroring Nexus server - either per the whole build, or
> just standalone, might be very beneficial for the build time/failure rates.
>
> Cos
I like to have it both the -trunk and -trunk-packages, too.
Problem is how to start/stop it, since a build will fail almost immediatly if one restarts the nexus while a compile is running.
One approach is to restart it manually. So I renamed the job to nexus-restart.
If no job is running feel free to restart nexus (please wait ~10sec after this before starting jobs depending on it).
The job uses the Node and Label parameter plugin to execute it on all docker-slaves.
The build jobs are now changed to use the nexus containes. Hopefully they work now ;-)
Olaf
PS: The build failures are gone after updating our infra.
Re: Bigtop-trunk is all blue
Posted by Konstantin Boudnik <co...@apache.org>.
On Sat, Jan 09, 2016 at 12:25PM, Evans Ye wrote:
> Bigtop-trunk-packages is about to back in business, I guess.
>
> We just need to comment out:
>
> #docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS
> --net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \
> #bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean ; ./gradlew
> configure-nexus $COMPONENTS-pkg'
>
> It seems running well now:
> http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>
>
>
> Hey Olaf,
> I've a question, can't we start a nexus server on CI master for permanent
> and then add the repository for each build?
I think having a local mirroring Nexus server - either per the whole build, or
just standalone, might be very beneficial for the build time/failure rates.
Cos
> 2016-01-09 4:32 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
> > On Fri, Jan 08, 2016 at 09:22PM, Olaf Flebbe wrote:
> > > Hi
> > >
> > > I am talking about Bigtop-trunk not Bigtop-trunk-packages.
> > > http://ci.bigtop.apache.org/job/Bigtop-trunk/
> >
> > Oh, my bad, sorry. Packages are a part of our CI pipeline and serves a
> > different purpose of course.
> >
> > > That's the all-in-one thingy- opposed to the great visualisation of
> > packages
> > > vs operating system matrix.
> > >
> > > I tried several aproaches to create a repository out of
> > Bigtop-trunk-packags
> > > but it turned out really complicated. Bigtop-trunk already generates
> >
> > Yeah, there's a bit hurdle with that. I have shared a link to a couple of
> > scripts I've put together for a project I am doing, which essentially
> > allows
> > to collect together all the packages from the dockerized build and then
> > create
> > the repos.
> >
> > For the release (RC or whatever) we should use the Bigtop-trunk of course -
> > it's a one-off build essentially. I think this is what Evans have done to
> > build 1.0.0 binary artifacts.
> >
> > Thanks!
> > Cos
> >
> > > repositories, so I suppose we should carve 1.1 repositories out of
> > > Bigtop-trunk rather assembling back the artifacts of
> > Bigtop-trunk-packages.
> > > See for instance:
> > >
> > http://ci.bigtop.apache.org/job/Bigtop-trunk/BUILD_ENVIRONMENTS=debian-8,label=docker-slave/lastSuccessfulBuild/artifact/output/apt/
> > >
> > > Olaf
> > >
> > >
> > > stantin Boudnik <co...@apache.org>:
> > > >
> > > >
> > > > I might be color-challenged %) then
> > > >
> > http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > >
> > > > cos
> > > >
> > > > On Fri, Jan 08, 2016 at 06:43PM, Olaf Flebbe wrote:
> > > >> Hi,
> > > >>
> > > >> updating the CI Infrastructure made Bigtop-trunk all blue.
> > > >>
> > > >> I will disable the nexus workaroung now for Bigtop-trunk-packages . A
> > new bigtop/slaves:trunk-opensuse-13.2 image is pushed
> > > >>
> > > >>
> > > >> Olaf
> > > >
> > > >
> > >
> >
> >
> >
Re: Bigtop-trunk is all blue
Posted by Evans Ye <ev...@apache.org>.
Bigtop-trunk-packages is about to back in business, I guess.
We just need to comment out:
#docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS
--net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \
#bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean ; ./gradlew
configure-nexus $COMPONENTS-pkg'
It seems running well now:
http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
Hey Olaf,
I've a question, can't we start a nexus server on CI master for permanent
and then add the repository for each build?
2016-01-09 4:32 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> On Fri, Jan 08, 2016 at 09:22PM, Olaf Flebbe wrote:
> > Hi
> >
> > I am talking about Bigtop-trunk not Bigtop-trunk-packages.
> > http://ci.bigtop.apache.org/job/Bigtop-trunk/
>
> Oh, my bad, sorry. Packages are a part of our CI pipeline and serves a
> different purpose of course.
>
> > That's the all-in-one thingy- opposed to the great visualisation of
> packages
> > vs operating system matrix.
> >
> > I tried several aproaches to create a repository out of
> Bigtop-trunk-packags
> > but it turned out really complicated. Bigtop-trunk already generates
>
> Yeah, there's a bit hurdle with that. I have shared a link to a couple of
> scripts I've put together for a project I am doing, which essentially
> allows
> to collect together all the packages from the dockerized build and then
> create
> the repos.
>
> For the release (RC or whatever) we should use the Bigtop-trunk of course -
> it's a one-off build essentially. I think this is what Evans have done to
> build 1.0.0 binary artifacts.
>
> Thanks!
> Cos
>
> > repositories, so I suppose we should carve 1.1 repositories out of
> > Bigtop-trunk rather assembling back the artifacts of
> Bigtop-trunk-packages.
> > See for instance:
> >
> http://ci.bigtop.apache.org/job/Bigtop-trunk/BUILD_ENVIRONMENTS=debian-8,label=docker-slave/lastSuccessfulBuild/artifact/output/apt/
> >
> > Olaf
> >
> >
> > stantin Boudnik <co...@apache.org>:
> > >
> > >
> > > I might be color-challenged %) then
> > >
> http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > >
> > > cos
> > >
> > > On Fri, Jan 08, 2016 at 06:43PM, Olaf Flebbe wrote:
> > >> Hi,
> > >>
> > >> updating the CI Infrastructure made Bigtop-trunk all blue.
> > >>
> > >> I will disable the nexus workaroung now for Bigtop-trunk-packages . A
> new bigtop/slaves:trunk-opensuse-13.2 image is pushed
> > >>
> > >>
> > >> Olaf
> > >
> > >
> >
>
>
>
Re: Bigtop-trunk is all blue
Posted by Konstantin Boudnik <co...@apache.org>.
On Fri, Jan 08, 2016 at 09:22PM, Olaf Flebbe wrote:
> Hi
>
> I am talking about Bigtop-trunk not Bigtop-trunk-packages.
> http://ci.bigtop.apache.org/job/Bigtop-trunk/
Oh, my bad, sorry. Packages are a part of our CI pipeline and serves a
different purpose of course.
> That's the all-in-one thingy- opposed to the great visualisation of packages
> vs operating system matrix.
>
> I tried several aproaches to create a repository out of Bigtop-trunk-packags
> but it turned out really complicated. Bigtop-trunk already generates
Yeah, there's a bit hurdle with that. I have shared a link to a couple of
scripts I've put together for a project I am doing, which essentially allows
to collect together all the packages from the dockerized build and then create
the repos.
For the release (RC or whatever) we should use the Bigtop-trunk of course -
it's a one-off build essentially. I think this is what Evans have done to
build 1.0.0 binary artifacts.
Thanks!
Cos
> repositories, so I suppose we should carve 1.1 repositories out of
> Bigtop-trunk rather assembling back the artifacts of Bigtop-trunk-packages.
> See for instance:
> http://ci.bigtop.apache.org/job/Bigtop-trunk/BUILD_ENVIRONMENTS=debian-8,label=docker-slave/lastSuccessfulBuild/artifact/output/apt/
>
> Olaf
>
>
> stantin Boudnik <co...@apache.org>:
> >
> >
> > I might be color-challenged %) then
> > http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >
> > cos
> >
> > On Fri, Jan 08, 2016 at 06:43PM, Olaf Flebbe wrote:
> >> Hi,
> >>
> >> updating the CI Infrastructure made Bigtop-trunk all blue.
> >>
> >> I will disable the nexus workaroung now for Bigtop-trunk-packages . A new bigtop/slaves:trunk-opensuse-13.2 image is pushed
> >>
> >>
> >> Olaf
> >
> >
>
Re: Bigtop-trunk is all blue
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi
I am talking about Bigtop-trunk not Bigtop-trunk-packages.
http://ci.bigtop.apache.org/job/Bigtop-trunk/
That's the all-in-one thingy- opposed to the great visualisation of packages vs operating system matrix.
I tried several aproaches to create a repository out of Bigtop-trunk-packags but it turned out really complicated.
Bigtop-trunk already generates repositories, so I suppose we should carve 1.1 repositories out of Bigtop-trunk rather assembling back the artifacts of Bigtop-trunk-packages.
See for instance:
http://ci.bigtop.apache.org/job/Bigtop-trunk/BUILD_ENVIRONMENTS=debian-8,label=docker-slave/lastSuccessfulBuild/artifact/output/apt/
Olaf
stantin Boudnik <co...@apache.org>:
>
>
> I might be color-challenged %) then
> http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>
> cos
>
> On Fri, Jan 08, 2016 at 06:43PM, Olaf Flebbe wrote:
>> Hi,
>>
>> updating the CI Infrastructure made Bigtop-trunk all blue.
>>
>> I will disable the nexus workaroung now for Bigtop-trunk-packages . A new bigtop/slaves:trunk-opensuse-13.2 image is pushed
>>
>>
>> Olaf
>
>
Re: Bigtop-trunk is all blue
Posted by Konstantin Boudnik <co...@apache.org>.
I might be color-challenged %) then
http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
cos
On Fri, Jan 08, 2016 at 06:43PM, Olaf Flebbe wrote:
> Hi,
>
> updating the CI Infrastructure made Bigtop-trunk all blue.
>
> I will disable the nexus workaroung now for Bigtop-trunk-packages . A new bigtop/slaves:trunk-opensuse-13.2 image is pushed
>
>
> Olaf