You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Gavin McDonald <gm...@apache.org> on 2021/06/30 18:03:15 UTC
[JENKINS] - New Maven Controller for the project
Hi Maven folks.
Infra has decided to separate off the Maven build jobs from
ci-builds.apache.org over to its very own Jenkins Controller and Agents.
This means that Maven now has a dedicated Jenkins environment for itself.
It
also means that no other projects build jobs can build on the Maven nodes;
and
then Maven jobs will no longer be able to build on the ci-builds jobs.
Your new Controller is set up as https://ci-maven.apache.org and all Maven
Committers
can login via LDAP and create jobs.
At the time of writing, there is one node/agent attached but I am building
4 more - all
Ubuntu 20.04 and based in our Azure account.
We can automagically move all your jobs over from ci-builds to ci-maven - I
just need someone to tell me go ahead and do it.
In the meantime, feel free to have a test. The remaining 4 agents will be
online
by tomorrow. We will review after a month if 5 is enough nodes.
As with other projects having their own dedicated controller, who have
taken advantage
of this isolation by having some nodes/agents given to the project as a
'targeted donation'
so someone here may know of a Company will to donate 5 - 10 or more nodes
specifically
for Maven Jenkins environment. Infra can afford to hand you over 5 right
now.
Let me know if you have any questions, otherwise let me know when I can
make the
transfer of your jobs.
Thanks
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
ok Gavin
Thanks for your feedback
It's what I wanted to be sure, I didn't want to break something you are
maintaining
On Wed, Jul 21, 2021 at 6:47 PM Gavin McDonald <gm...@apache.org> wrote:
> Hi Arnaud
>
> Surem feel free to tweak the node configs directly, these configs are not
> in version control or puppet.
>
> On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
>> I am looking at our builds and I try to understand why our agents are
>> often disconnected during the builds.
>> We have in general a stacktrace like
>>
>> maven6 was marked offline: Connection was broken: java.io.IOException: Pipe closed after 0 cycles
>> at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
>> at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
>> at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
>> at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
>> at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
>> at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
>> at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
>> at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>>
>>
>>
>> As far I can see we are using 16Gb "hosts" for linux agents
>>
>> Something very strange is that the jenkins agent (this small component
>> doing the link between the build host and the controller) is configured
>> with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
>> (even more because of the non-heap)
>> This one in general should require in general really less. 1Gb is already
>> a lot from my exp.
>> Due to this, the OS can see it has the biggest process on the host and
>> decide to kill it when the rest of the memory is used by the build.
>> I think we should decrease this value.
>> (I can do it but I don't know how was configured the ci.apache.org
>> agents and I would like to not add more issue if this setting was here in
>> the past
>>
>> I don't think it is the root cause of our instabilities (at least all)
>> and there is something else I have to find but it's a cheap fix to try
>>
>> FYI our agents VMs are ~like this today:
>>
>> - Java
>> + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
>> + Vendor: Oracle Corporation
>> + Version: 1.8.0_291
>> + Maximum memory: 7.67 GB (8232370176)
>> + Allocated memory: 7.67 GB (8232370176)
>> + Free memory: 6.03 GB (6470953760)
>> + In-use memory: 1.64 GB (1761416416)
>> + GC strategy: ParallelGC
>> + Available CPUs: 4
>>
>> 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free mem
>> is high)
>>
>> I would be in favor to try to launch them with -Xms128m
>> -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>>
>> I think it's enough customization to start with
>>
>> Cheers
>>
>> On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com>
>> wrote:
>>
>>> I am not sure about the setup
>>> AFAICS we don't use any JDK installer (
>>> https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
>>> different JDKs are supposed to be installed directly on the agent ?
>>> I am not sure how it was done on the previous environment
>>>
>>> On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org>
>>> wrote:
>>>
>>>> The new CI system has the following issue:
>>>>
>>>> /home/jenkins/tools/java/latest1.7/bin/java: not found
>>>>
>>>>
>>>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
>>>> wrote:
>>>>
>>>> > Hi Maven folks.
>>>> >
>>>> > Infra has decided to separate off the Maven build jobs from
>>>> > ci-builds.apache.org over to its very own Jenkins Controller and
>>>> Agents.
>>>> >
>>>> > This means that Maven now has a dedicated Jenkins environment for
>>>> itself.
>>>> > It
>>>> > also means that no other projects build jobs can build on the Maven
>>>> nodes;
>>>> > and
>>>> > then Maven jobs will no longer be able to build on the ci-builds
>>>> jobs.
>>>> >
>>>> > Your new Controller is set up as https://ci-maven.apache.org and all
>>>> Maven
>>>> > Committers
>>>> > can login via LDAP and create jobs.
>>>> >
>>>> > At the time of writing, there is one node/agent attached but I am
>>>> building
>>>> > 4 more - all
>>>> > Ubuntu 20.04 and based in our Azure account.
>>>> >
>>>> > We can automagically move all your jobs over from ci-builds to
>>>> ci-maven - I
>>>> > just need someone to tell me go ahead and do it.
>>>> >
>>>> > In the meantime, feel free to have a test. The remaining 4 agents
>>>> will be
>>>> > online
>>>> > by tomorrow. We will review after a month if 5 is enough nodes.
>>>> >
>>>> > As with other projects having their own dedicated controller, who have
>>>> > taken advantage
>>>> > of this isolation by having some nodes/agents given to the project as
>>>> a
>>>> > 'targeted donation'
>>>> > so someone here may know of a Company will to donate 5 - 10 or more
>>>> nodes
>>>> > specifically
>>>> > for Maven Jenkins environment. Infra can afford to hand you over 5
>>>> right
>>>> > now.
>>>> >
>>>> > Let me know if you have any questions, otherwise let me know when I
>>>> can
>>>> > make the
>>>> > transfer of your jobs.
>>>> >
>>>> > Thanks
>>>> >
>>>> > --
>>>> >
>>>> > *Gavin McDonald*
>>>> > Systems Administrator
>>>> > ASF Infrastructure Team
>>>> >
>>>>
>>>
>>>
>>> --
>>> Arnaud Héritier
>>> Twitter/Skype : aheritier
>>>
>>
>>
>> --
>> Arnaud Héritier
>> Twitter/Skype : aheritier
>>
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi Arnaud
Surem feel free to tweak the node configs directly, these configs are not
in version control or puppet.
On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com> wrote:
> I am looking at our builds and I try to understand why our agents are
> often disconnected during the builds.
> We have in general a stacktrace like
>
> maven6 was marked offline: Connection was broken: java.io.IOException: Pipe closed after 0 cycles
> at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>
>
>
> As far I can see we are using 16Gb "hosts" for linux agents
>
> Something very strange is that the jenkins agent (this small component
> doing the link between the build host and the controller) is configured
> with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
> (even more because of the non-heap)
> This one in general should require in general really less. 1Gb is already
> a lot from my exp.
> Due to this, the OS can see it has the biggest process on the host and
> decide to kill it when the rest of the memory is used by the build.
> I think we should decrease this value.
> (I can do it but I don't know how was configured the ci.apache.org agents
> and I would like to not add more issue if this setting was here in the past
>
> I don't think it is the root cause of our instabilities (at least all) and
> there is something else I have to find but it's a cheap fix to try
>
> FYI our agents VMs are ~like this today:
>
> - Java
> + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> + Vendor: Oracle Corporation
> + Version: 1.8.0_291
> + Maximum memory: 7.67 GB (8232370176)
> + Allocated memory: 7.67 GB (8232370176)
> + Free memory: 6.03 GB (6470953760)
> + In-use memory: 1.64 GB (1761416416)
> + GC strategy: ParallelGC
> + Available CPUs: 4
>
> 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free mem
> is high)
>
> I would be in favor to try to launch them with -Xms128m
> -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>
> I think it's enough customization to start with
>
> Cheers
>
> On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
>> I am not sure about the setup
>> AFAICS we don't use any JDK installer (
>> https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
>> different JDKs are supposed to be installed directly on the agent ?
>> I am not sure how it was done on the previous environment
>>
>> On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org>
>> wrote:
>>
>>> The new CI system has the following issue:
>>>
>>> /home/jenkins/tools/java/latest1.7/bin/java: not found
>>>
>>>
>>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>>>
>>>
>>>
>>> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
>>> wrote:
>>>
>>> > Hi Maven folks.
>>> >
>>> > Infra has decided to separate off the Maven build jobs from
>>> > ci-builds.apache.org over to its very own Jenkins Controller and
>>> Agents.
>>> >
>>> > This means that Maven now has a dedicated Jenkins environment for
>>> itself.
>>> > It
>>> > also means that no other projects build jobs can build on the Maven
>>> nodes;
>>> > and
>>> > then Maven jobs will no longer be able to build on the ci-builds jobs.
>>> >
>>> > Your new Controller is set up as https://ci-maven.apache.org and all
>>> Maven
>>> > Committers
>>> > can login via LDAP and create jobs.
>>> >
>>> > At the time of writing, there is one node/agent attached but I am
>>> building
>>> > 4 more - all
>>> > Ubuntu 20.04 and based in our Azure account.
>>> >
>>> > We can automagically move all your jobs over from ci-builds to
>>> ci-maven - I
>>> > just need someone to tell me go ahead and do it.
>>> >
>>> > In the meantime, feel free to have a test. The remaining 4 agents will
>>> be
>>> > online
>>> > by tomorrow. We will review after a month if 5 is enough nodes.
>>> >
>>> > As with other projects having their own dedicated controller, who have
>>> > taken advantage
>>> > of this isolation by having some nodes/agents given to the project as a
>>> > 'targeted donation'
>>> > so someone here may know of a Company will to donate 5 - 10 or more
>>> nodes
>>> > specifically
>>> > for Maven Jenkins environment. Infra can afford to hand you over 5
>>> right
>>> > now.
>>> >
>>> > Let me know if you have any questions, otherwise let me know when I can
>>> > make the
>>> > transfer of your jobs.
>>> >
>>> > Thanks
>>> >
>>> > --
>>> >
>>> > *Gavin McDonald*
>>> > Systems Administrator
>>> > ASF Infrastructure Team
>>> >
>>>
>>
>>
>> --
>> Arnaud Héritier
>> Twitter/Skype : aheritier
>>
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
To close this thread, the disconnection of linux agents was a side effect
of the kernel settings which were using a too long tcp timeout
Gavin applied the settings recommended in this doc (
https://support.cloudbees.com/hc/en-us/articles/115001369667-Dedicated-SSH-agent-gets-disconnected
) and it solved this issue
sysctl -w net.ipv4.tcp_keepalive_time=120 sysctl -w
net.ipv4.tcp_keepalive_intvl=30 sysctl -w net.ipv4.tcp_keepalive_probes=8
sysctl -w net.ipv4.tcp_fin_timeout=30
Thanks a lot Gavin for your help
On Tue, Jul 27, 2021 at 10:48 AM Arnaud Héritier <ah...@gmail.com>
wrote:
> 👍 thanks
> As discussed on Slack I will open.a support case on CloudBees side to
> study the instability issue of linux agents.
> I will verify that there is nothing wrong in the new setup (but I found
> nothing bad personally which could create such issue)
> The major change when we compare ci-builds and ci-maven environments is
> that our agents are now running on Azure and sadly I heard about similar
> issues in others communities using Azure like Jenkins
> I will check if we can find a solution or if we can give enough details to
> Gavin to open a case on Azure side too.
> If we really don't find any solution with Azure we'll see with Gavin to
> deploy our agents somewhere else (but let's try to give a chance to Azure
> first)
>
> Cheers
>
>
>
> On Tue, Jul 27, 2021 at 10:37 AM Gavin McDonald <gm...@apache.org>
> wrote:
>
>> On Tue, Jul 27, 2021 at 10:18 AM Arnaud Héritier <ah...@gmail.com>
>> wrote:
>>
>> > Gavin, these JDKs are only for build agents, right ?
>> > Tibor was asking for the JVM used to host Tomcat/Jenkins.
>> > (And I suppose that the controller part is templatised)
>> >
>>
>> Oh right, sorry, yes the client controllers use a system openjdk 8
>>
>>
>> https://github.com/apache/infrastructure-p6/blob/production/modules/jenkins_client_master/manifests/init.pp
>>
>>
>> > On Mon, Jul 26, 2021 at 1:06 PM Gavin McDonald <gm...@apache.org>
>> > wrote:
>> >
>> >> There are MANY JDKS installed already. Oracle JDKs OpenJDKs
>> AdoptOpenJDKs
>> >> - they are all already there.
>> >>
>> >>
>> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>> >>
>> >> On Mon, Jul 26, 2021 at 11:47 AM Arnaud Héritier <ah...@gmail.com>
>> >> wrote:
>> >>
>> >> > It has to be discussed with infra.
>> >> > I am not sure which distro is used.
>> >> > It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
>> >> > something else than Java 8 - no comment)
>> >> > I don't have the feeling for now (with the data I reviewed) that
>> it's a
>> >> > memory / GC issue (it could create disconnections under high load)
>> >> > Here the stability issue occurs even when the controller does nothing
>> >> (the
>> >> > controller cannot ping the agent or vice and versa) and it seems to
>> >> impact
>> >> > more the linux agents than the windows ones (it's a pity)
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <
>> tibordigana@apache.org>
>> >> > wrote:
>> >> >
>> >> >> Can you install AdoptOpenJdk for the Jenkins controller?
>> >> >> It contains Eclipse OpenJ9 Garbage Collector and it significantly
>> >> >> decreases
>> >> >> memory consumption of the application due to the meta space goes to
>> the
>> >> >> disk.
>> >> >> You should save 40 - 75% out of 3GB.
>> >> >> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most
>> >> memory.
>> >> >>
>> >> >> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <
>> aheritier@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >> > yes for the controller it depends of its size (number of jobs and
>> >> types
>> >> >> of
>> >> >> > jobs) but here we are fine it seems with our 3Gb
>> >> >> >
>> >> >> > * Java
>> >> >> > - Version: 1.8.0_292
>> >> >> > - Maximum memory: 3.00 GB (3221225472)
>> >> >> > - Allocated memory: 3.00 GB (3221225472)
>> >> >> > - Free memory: 750.15 MB (786591664)
>> >> >> > - In-use memory: 2.27 GB (2434633808)
>> >> >> > - GC strategy: G1
>> >> >> > - Available CPUs: 2
>> >> >> >
>> >> >> > For agents I reduced the memory allocated to the agent process
>> but it
>> >> >> > doesn't help much (it seems - even if it is still a good thing to
>> do)
>> >> >> >
>> >> >> > What is strange is that I see our agents sometimes disconnected
>> even
>> >> >> when
>> >> >> > we have no activity on the jenkins controller
>> >> >> >
>> >> >> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get
>> access
>> >> to
>> >> >> its
>> >> >> > logs
>> >> >> >
>> >> >> > In general the connection lost is detected by what we call the
>> >> >> PingThread (
>> >> >> >
>> >> >> >
>> >> >>
>> >>
>> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
>> >> >> > ) but not only
>> >> >> >
>> >> >> > https://ci-maven.apache.org/log/all
>> >> >> >
>> >> >> > For example it was few minutes ago we got 3 agents disconnected
>> while
>> >> >> > nothing was running
>> >> >> >
>> >> >> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
>> >> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> >> > Ping failed. Terminating the channel maven4.
>> >> >> > java.util.concurrent.TimeoutException: Ping started at
>> 1626936861769
>> >> >> hasn't
>> >> >> > completed by 1626937101769
>> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> >> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
>> >> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> >> > Ping failed. Terminating the channel maven3.
>> >> >> > java.util.concurrent.TimeoutException: Ping started at
>> 1626936861777
>> >> >> hasn't
>> >> >> > completed by 1626937101778
>> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> >> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
>> >> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> >> > Ping failed. Terminating the channel maven5.
>> >> >> > java.util.concurrent.TimeoutException: Ping started at
>> 1626936861982
>> >> >> hasn't
>> >> >> > completed by 1626937101983
>> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> >> >
>> >> >> > @Gavin McDonald <gm...@apache.org> In terms of network, is it
>> >> the
>> >> >> same
>> >> >> > environment we use today compared to the ci-builds.apache.org
>> >> >> environment
>> >> >> > ?
>> >> >> >
>> >> >> >
>> >> >> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <
>> >> tibordigana@apache.org>
>> >> >> > wrote:
>> >> >> >
>> >> >> > > In my company, I also used 1GB for Xmx of Java Heap for the
>> Jenkins
>> >> >> JVM
>> >> >> > and
>> >> >> > > it was enough.
>> >> >> > > The subprocesses like Maven need to have much more memory to
>> >> allocate
>> >> >> for
>> >> >> > > themself rather than Jenkins JVM.
>> >> >> > > T
>> >> >> > >
>> >> >> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <
>> >> aheritier@gmail.com>
>> >> >> > > wrote:
>> >> >> > >
>> >> >> > > > I am looking at our builds and I try to understand why our
>> agents
>> >> >> are
>> >> >> > > often
>> >> >> > > > disconnected during the builds.
>> >> >> > > > We have in general a stacktrace like
>> >> >> > > >
>> >> >> > > > maven6 was marked offline: Connection was broken:
>> >> >> java.io.IOException:
>> >> >> > > > Pipe closed after 0 cycles
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
>> >> >> > > > at
>> >> >> > > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > As far I can see we are using 16Gb "hosts" for linux agents
>> >> >> > > >
>> >> >> > > > Something very strange is that the jenkins agent (this small
>> >> >> component
>> >> >> > > > doing the link between the build host and the controller) is
>> >> >> configured
>> >> >> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the
>> >> server
>> >> >> mem
>> >> >> > > > (even more because of the non-heap)
>> >> >> > > > This one in general should require in general really less.
>> 1Gb is
>> >> >> > > already a
>> >> >> > > > lot from my exp.
>> >> >> > > > Due to this, the OS can see it has the biggest process on the
>> >> host
>> >> >> and
>> >> >> > > > decide to kill it when the rest of the memory is used by the
>> >> build.
>> >> >> > > > I think we should decrease this value.
>> >> >> > > > (I can do it but I don't know how was configured the
>> >> ci.apache.org
>> >> >> > > agents
>> >> >> > > > and I would like to not add more issue if this setting was
>> here
>> >> in
>> >> >> the
>> >> >> > > past
>> >> >> > > >
>> >> >> > > > I don't think it is the root cause of our instabilities (at
>> least
>> >> >> all)
>> >> >> > > and
>> >> >> > > > there is something else I have to find but it's a cheap fix to
>> >> try
>> >> >> > > >
>> >> >> > > > FYI our agents VMs are ~like this today:
>> >> >> > > >
>> >> >> > > > - Java
>> >> >> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
>> >> >> > > > + Vendor: Oracle Corporation
>> >> >> > > > + Version: 1.8.0_291
>> >> >> > > > + Maximum memory: 7.67 GB (8232370176)
>> >> >> > > > + Allocated memory: 7.67 GB (8232370176)
>> >> >> > > > + Free memory: 6.03 GB (6470953760)
>> >> >> > > > + In-use memory: 1.64 GB (1761416416)
>> >> >> > > > + GC strategy: ParallelGC
>> >> >> > > > + Available CPUs: 4
>> >> >> > > >
>> >> >> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as
>> the
>> >> >> Free
>> >> >> > mem
>> >> >> > > > is high)
>> >> >> > > >
>> >> >> > > > I would be in favor to try to launch them with -Xms128m
>> >> >> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>> >> >> > > >
>> >> >> > > > I think it's enough customization to start with
>> >> >> > > >
>> >> >> > > > Cheers
>> >> >> > > >
>> >> >> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <
>> >> >> aheritier@gmail.com>
>> >> >> > > > wrote:
>> >> >> > > >
>> >> >> > > > > I am not sure about the setup
>> >> >> > > > > AFAICS we don't use any JDK installer (
>> >> >> > > > > https://ci-maven.apache.org/configureTools/ ) thus I
>> suppose
>> >> that
>> >> >> > the
>> >> >> > > > > different JDKs are supposed to be installed directly on the
>> >> agent
>> >> >> ?
>> >> >> > > > > I am not sure how it was done on the previous environment
>> >> >> > > > >
>> >> >> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
>> >> >> tibordigana@apache.org
>> >> >> > >
>> >> >> > > > > wrote:
>> >> >> > > > >
>> >> >> > > > >> The new CI system has the following issue:
>> >> >> > > > >>
>> >> >> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
>> >> >> > > > >>
>> >> >> > > > >>
>> >> >> > > > >>
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>> >> >> > > > >>
>> >> >> > > > >>
>> >> >> > > > >>
>> >> >> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
>> >> >> > gmcdonald@apache.org>
>> >> >> > > > >> wrote:
>> >> >> > > > >>
>> >> >> > > > >> > Hi Maven folks.
>> >> >> > > > >> >
>> >> >> > > > >> > Infra has decided to separate off the Maven build jobs
>> from
>> >> >> > > > >> > ci-builds.apache.org over to its very own Jenkins
>> >> Controller
>> >> >> and
>> >> >> > > > >> Agents.
>> >> >> > > > >> >
>> >> >> > > > >> > This means that Maven now has a dedicated Jenkins
>> >> environment
>> >> >> for
>> >> >> > > > >> itself.
>> >> >> > > > >> > It
>> >> >> > > > >> > also means that no other projects build jobs can build on
>> >> the
>> >> >> > Maven
>> >> >> > > > >> nodes;
>> >> >> > > > >> > and
>> >> >> > > > >> > then Maven jobs will no longer be able to build on the
>> >> >> ci-builds
>> >> >> > > > jobs.
>> >> >> > > > >> >
>> >> >> > > > >> > Your new Controller is set up as
>> >> https://ci-maven.apache.org
>> >> >> and
>> >> >> > > all
>> >> >> > > > >> Maven
>> >> >> > > > >> > Committers
>> >> >> > > > >> > can login via LDAP and create jobs.
>> >> >> > > > >> >
>> >> >> > > > >> > At the time of writing, there is one node/agent attached
>> >> but I
>> >> >> am
>> >> >> > > > >> building
>> >> >> > > > >> > 4 more - all
>> >> >> > > > >> > Ubuntu 20.04 and based in our Azure account.
>> >> >> > > > >> >
>> >> >> > > > >> > We can automagically move all your jobs over from
>> ci-builds
>> >> to
>> >> >> > > > ci-maven
>> >> >> > > > >> - I
>> >> >> > > > >> > just need someone to tell me go ahead and do it.
>> >> >> > > > >> >
>> >> >> > > > >> > In the meantime, feel free to have a test. The remaining
>> 4
>> >> >> agents
>> >> >> > > will
>> >> >> > > > >> be
>> >> >> > > > >> > online
>> >> >> > > > >> > by tomorrow. We will review after a month if 5 is enough
>> >> nodes.
>> >> >> > > > >> >
>> >> >> > > > >> > As with other projects having their own dedicated
>> >> controller,
>> >> >> who
>> >> >> > > have
>> >> >> > > > >> > taken advantage
>> >> >> > > > >> > of this isolation by having some nodes/agents given to
>> the
>> >> >> project
>> >> >> > > as
>> >> >> > > > a
>> >> >> > > > >> > 'targeted donation'
>> >> >> > > > >> > so someone here may know of a Company will to donate 5 -
>> 10
>> >> or
>> >> >> > more
>> >> >> > > > >> nodes
>> >> >> > > > >> > specifically
>> >> >> > > > >> > for Maven Jenkins environment. Infra can afford to hand
>> you
>> >> >> over 5
>> >> >> > > > right
>> >> >> > > > >> > now.
>> >> >> > > > >> >
>> >> >> > > > >> > Let me know if you have any questions, otherwise let me
>> know
>> >> >> when
>> >> >> > I
>> >> >> > > > can
>> >> >> > > > >> > make the
>> >> >> > > > >> > transfer of your jobs.
>> >> >> > > > >> >
>> >> >> > > > >> > Thanks
>> >> >> > > > >> >
>> >> >> > > > >> > --
>> >> >> > > > >> >
>> >> >> > > > >> > *Gavin McDonald*
>> >> >> > > > >> > Systems Administrator
>> >> >> > > > >> > ASF Infrastructure Team
>> >> >> > > > >> >
>> >> >> > > > >>
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > > --
>> >> >> > > > > Arnaud Héritier
>> >> >> > > > > Twitter/Skype : aheritier
>> >> >> > > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > --
>> >> >> > > > Arnaud Héritier
>> >> >> > > > Twitter/Skype : aheritier
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Arnaud Héritier
>> >> >> > Twitter/Skype : aheritier
>> >> >> >
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Arnaud Héritier
>> >> > Twitter/Skype : aheritier
>> >> >
>> >>
>> >>
>> >> --
>> >>
>> >> *Gavin McDonald*
>> >> Systems Administrator
>> >> ASF Infrastructure Team
>> >>
>> >
>> >
>> > --
>> > Arnaud Héritier
>> > Twitter/Skype : aheritier
>> >
>>
>>
>> --
>>
>> *Gavin McDonald*
>> Systems Administrator
>> ASF Infrastructure Team
>>
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
Arnaud Héritier
Twitter/GitHub/... : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
👍 thanks
As discussed on Slack I will open.a support case on CloudBees side to study
the instability issue of linux agents.
I will verify that there is nothing wrong in the new setup (but I found
nothing bad personally which could create such issue)
The major change when we compare ci-builds and ci-maven environments is
that our agents are now running on Azure and sadly I heard about similar
issues in others communities using Azure like Jenkins
I will check if we can find a solution or if we can give enough details to
Gavin to open a case on Azure side too.
If we really don't find any solution with Azure we'll see with Gavin to
deploy our agents somewhere else (but let's try to give a chance to Azure
first)
Cheers
On Tue, Jul 27, 2021 at 10:37 AM Gavin McDonald <gm...@apache.org>
wrote:
> On Tue, Jul 27, 2021 at 10:18 AM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
> > Gavin, these JDKs are only for build agents, right ?
> > Tibor was asking for the JVM used to host Tomcat/Jenkins.
> > (And I suppose that the controller part is templatised)
> >
>
> Oh right, sorry, yes the client controllers use a system openjdk 8
>
>
> https://github.com/apache/infrastructure-p6/blob/production/modules/jenkins_client_master/manifests/init.pp
>
>
> > On Mon, Jul 26, 2021 at 1:06 PM Gavin McDonald <gm...@apache.org>
> > wrote:
> >
> >> There are MANY JDKS installed already. Oracle JDKs OpenJDKs
> AdoptOpenJDKs
> >> - they are all already there.
> >>
> >>
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>
> >> On Mon, Jul 26, 2021 at 11:47 AM Arnaud Héritier <ah...@gmail.com>
> >> wrote:
> >>
> >> > It has to be discussed with infra.
> >> > I am not sure which distro is used.
> >> > It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
> >> > something else than Java 8 - no comment)
> >> > I don't have the feeling for now (with the data I reviewed) that it's
> a
> >> > memory / GC issue (it could create disconnections under high load)
> >> > Here the stability issue occurs even when the controller does nothing
> >> (the
> >> > controller cannot ping the agent or vice and versa) and it seems to
> >> impact
> >> > more the linux agents than the windows ones (it's a pity)
> >> >
> >> >
> >> >
> >> > On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <tibordigana@apache.org
> >
> >> > wrote:
> >> >
> >> >> Can you install AdoptOpenJdk for the Jenkins controller?
> >> >> It contains Eclipse OpenJ9 Garbage Collector and it significantly
> >> >> decreases
> >> >> memory consumption of the application due to the meta space goes to
> the
> >> >> disk.
> >> >> You should save 40 - 75% out of 3GB.
> >> >> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most
> >> memory.
> >> >>
> >> >> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <aheritier@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >> > yes for the controller it depends of its size (number of jobs and
> >> types
> >> >> of
> >> >> > jobs) but here we are fine it seems with our 3Gb
> >> >> >
> >> >> > * Java
> >> >> > - Version: 1.8.0_292
> >> >> > - Maximum memory: 3.00 GB (3221225472)
> >> >> > - Allocated memory: 3.00 GB (3221225472)
> >> >> > - Free memory: 750.15 MB (786591664)
> >> >> > - In-use memory: 2.27 GB (2434633808)
> >> >> > - GC strategy: G1
> >> >> > - Available CPUs: 2
> >> >> >
> >> >> > For agents I reduced the memory allocated to the agent process but
> it
> >> >> > doesn't help much (it seems - even if it is still a good thing to
> do)
> >> >> >
> >> >> > What is strange is that I see our agents sometimes disconnected
> even
> >> >> when
> >> >> > we have no activity on the jenkins controller
> >> >> >
> >> >> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get access
> >> to
> >> >> its
> >> >> > logs
> >> >> >
> >> >> > In general the connection lost is detected by what we call the
> >> >> PingThread (
> >> >> >
> >> >> >
> >> >>
> >>
> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
> >> >> > ) but not only
> >> >> >
> >> >> > https://ci-maven.apache.org/log/all
> >> >> >
> >> >> > For example it was few minutes ago we got 3 agents disconnected
> while
> >> >> > nothing was running
> >> >> >
> >> >> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
> >> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> >> > Ping failed. Terminating the channel maven4.
> >> >> > java.util.concurrent.TimeoutException: Ping started at
> 1626936861769
> >> >> hasn't
> >> >> > completed by 1626937101769
> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> >> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
> >> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> >> > Ping failed. Terminating the channel maven3.
> >> >> > java.util.concurrent.TimeoutException: Ping started at
> 1626936861777
> >> >> hasn't
> >> >> > completed by 1626937101778
> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> >> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
> >> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> >> > Ping failed. Terminating the channel maven5.
> >> >> > java.util.concurrent.TimeoutException: Ping started at
> 1626936861982
> >> >> hasn't
> >> >> > completed by 1626937101983
> >> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> >> >
> >> >> > @Gavin McDonald <gm...@apache.org> In terms of network, is it
> >> the
> >> >> same
> >> >> > environment we use today compared to the ci-builds.apache.org
> >> >> environment
> >> >> > ?
> >> >> >
> >> >> >
> >> >> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <
> >> tibordigana@apache.org>
> >> >> > wrote:
> >> >> >
> >> >> > > In my company, I also used 1GB for Xmx of Java Heap for the
> Jenkins
> >> >> JVM
> >> >> > and
> >> >> > > it was enough.
> >> >> > > The subprocesses like Maven need to have much more memory to
> >> allocate
> >> >> for
> >> >> > > themself rather than Jenkins JVM.
> >> >> > > T
> >> >> > >
> >> >> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <
> >> aheritier@gmail.com>
> >> >> > > wrote:
> >> >> > >
> >> >> > > > I am looking at our builds and I try to understand why our
> agents
> >> >> are
> >> >> > > often
> >> >> > > > disconnected during the builds.
> >> >> > > > We have in general a stacktrace like
> >> >> > > >
> >> >> > > > maven6 was marked offline: Connection was broken:
> >> >> java.io.IOException:
> >> >> > > > Pipe closed after 0 cycles
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> >> >> > > > at
> >> >> > > >
> >> >> >
> >> >>
> >>
> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> >> >> > > > at
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > As far I can see we are using 16Gb "hosts" for linux agents
> >> >> > > >
> >> >> > > > Something very strange is that the jenkins agent (this small
> >> >> component
> >> >> > > > doing the link between the build host and the controller) is
> >> >> configured
> >> >> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the
> >> server
> >> >> mem
> >> >> > > > (even more because of the non-heap)
> >> >> > > > This one in general should require in general really less. 1Gb
> is
> >> >> > > already a
> >> >> > > > lot from my exp.
> >> >> > > > Due to this, the OS can see it has the biggest process on the
> >> host
> >> >> and
> >> >> > > > decide to kill it when the rest of the memory is used by the
> >> build.
> >> >> > > > I think we should decrease this value.
> >> >> > > > (I can do it but I don't know how was configured the
> >> ci.apache.org
> >> >> > > agents
> >> >> > > > and I would like to not add more issue if this setting was here
> >> in
> >> >> the
> >> >> > > past
> >> >> > > >
> >> >> > > > I don't think it is the root cause of our instabilities (at
> least
> >> >> all)
> >> >> > > and
> >> >> > > > there is something else I have to find but it's a cheap fix to
> >> try
> >> >> > > >
> >> >> > > > FYI our agents VMs are ~like this today:
> >> >> > > >
> >> >> > > > - Java
> >> >> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> >> >> > > > + Vendor: Oracle Corporation
> >> >> > > > + Version: 1.8.0_291
> >> >> > > > + Maximum memory: 7.67 GB (8232370176)
> >> >> > > > + Allocated memory: 7.67 GB (8232370176)
> >> >> > > > + Free memory: 6.03 GB (6470953760)
> >> >> > > > + In-use memory: 1.64 GB (1761416416)
> >> >> > > > + GC strategy: ParallelGC
> >> >> > > > + Available CPUs: 4
> >> >> > > >
> >> >> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as
> the
> >> >> Free
> >> >> > mem
> >> >> > > > is high)
> >> >> > > >
> >> >> > > > I would be in favor to try to launch them with -Xms128m
> >> >> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
> >> >> > > >
> >> >> > > > I think it's enough customization to start with
> >> >> > > >
> >> >> > > > Cheers
> >> >> > > >
> >> >> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <
> >> >> aheritier@gmail.com>
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > I am not sure about the setup
> >> >> > > > > AFAICS we don't use any JDK installer (
> >> >> > > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose
> >> that
> >> >> > the
> >> >> > > > > different JDKs are supposed to be installed directly on the
> >> agent
> >> >> ?
> >> >> > > > > I am not sure how it was done on the previous environment
> >> >> > > > >
> >> >> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
> >> >> tibordigana@apache.org
> >> >> > >
> >> >> > > > > wrote:
> >> >> > > > >
> >> >> > > > >> The new CI system has the following issue:
> >> >> > > > >>
> >> >> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> >> >> > > > >>
> >> >> > > > >>
> >> >> > > > >>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> >> >> > > > >>
> >> >> > > > >>
> >> >> > > > >>
> >> >> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
> >> >> > gmcdonald@apache.org>
> >> >> > > > >> wrote:
> >> >> > > > >>
> >> >> > > > >> > Hi Maven folks.
> >> >> > > > >> >
> >> >> > > > >> > Infra has decided to separate off the Maven build jobs
> from
> >> >> > > > >> > ci-builds.apache.org over to its very own Jenkins
> >> Controller
> >> >> and
> >> >> > > > >> Agents.
> >> >> > > > >> >
> >> >> > > > >> > This means that Maven now has a dedicated Jenkins
> >> environment
> >> >> for
> >> >> > > > >> itself.
> >> >> > > > >> > It
> >> >> > > > >> > also means that no other projects build jobs can build on
> >> the
> >> >> > Maven
> >> >> > > > >> nodes;
> >> >> > > > >> > and
> >> >> > > > >> > then Maven jobs will no longer be able to build on the
> >> >> ci-builds
> >> >> > > > jobs.
> >> >> > > > >> >
> >> >> > > > >> > Your new Controller is set up as
> >> https://ci-maven.apache.org
> >> >> and
> >> >> > > all
> >> >> > > > >> Maven
> >> >> > > > >> > Committers
> >> >> > > > >> > can login via LDAP and create jobs.
> >> >> > > > >> >
> >> >> > > > >> > At the time of writing, there is one node/agent attached
> >> but I
> >> >> am
> >> >> > > > >> building
> >> >> > > > >> > 4 more - all
> >> >> > > > >> > Ubuntu 20.04 and based in our Azure account.
> >> >> > > > >> >
> >> >> > > > >> > We can automagically move all your jobs over from
> ci-builds
> >> to
> >> >> > > > ci-maven
> >> >> > > > >> - I
> >> >> > > > >> > just need someone to tell me go ahead and do it.
> >> >> > > > >> >
> >> >> > > > >> > In the meantime, feel free to have a test. The remaining 4
> >> >> agents
> >> >> > > will
> >> >> > > > >> be
> >> >> > > > >> > online
> >> >> > > > >> > by tomorrow. We will review after a month if 5 is enough
> >> nodes.
> >> >> > > > >> >
> >> >> > > > >> > As with other projects having their own dedicated
> >> controller,
> >> >> who
> >> >> > > have
> >> >> > > > >> > taken advantage
> >> >> > > > >> > of this isolation by having some nodes/agents given to the
> >> >> project
> >> >> > > as
> >> >> > > > a
> >> >> > > > >> > 'targeted donation'
> >> >> > > > >> > so someone here may know of a Company will to donate 5 -
> 10
> >> or
> >> >> > more
> >> >> > > > >> nodes
> >> >> > > > >> > specifically
> >> >> > > > >> > for Maven Jenkins environment. Infra can afford to hand
> you
> >> >> over 5
> >> >> > > > right
> >> >> > > > >> > now.
> >> >> > > > >> >
> >> >> > > > >> > Let me know if you have any questions, otherwise let me
> know
> >> >> when
> >> >> > I
> >> >> > > > can
> >> >> > > > >> > make the
> >> >> > > > >> > transfer of your jobs.
> >> >> > > > >> >
> >> >> > > > >> > Thanks
> >> >> > > > >> >
> >> >> > > > >> > --
> >> >> > > > >> >
> >> >> > > > >> > *Gavin McDonald*
> >> >> > > > >> > Systems Administrator
> >> >> > > > >> > ASF Infrastructure Team
> >> >> > > > >> >
> >> >> > > > >>
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > --
> >> >> > > > > Arnaud Héritier
> >> >> > > > > Twitter/Skype : aheritier
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > --
> >> >> > > > Arnaud Héritier
> >> >> > > > Twitter/Skype : aheritier
> >> >> > > >
> >> >> > >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Arnaud Héritier
> >> >> > Twitter/Skype : aheritier
> >> >> >
> >> >>
> >> >
> >> >
> >> > --
> >> > Arnaud Héritier
> >> > Twitter/Skype : aheritier
> >> >
> >>
> >>
> >> --
> >>
> >> *Gavin McDonald*
> >> Systems Administrator
> >> ASF Infrastructure Team
> >>
> >
> >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
On Tue, Jul 27, 2021 at 10:18 AM Arnaud Héritier <ah...@gmail.com>
wrote:
> Gavin, these JDKs are only for build agents, right ?
> Tibor was asking for the JVM used to host Tomcat/Jenkins.
> (And I suppose that the controller part is templatised)
>
Oh right, sorry, yes the client controllers use a system openjdk 8
https://github.com/apache/infrastructure-p6/blob/production/modules/jenkins_client_master/manifests/init.pp
> On Mon, Jul 26, 2021 at 1:06 PM Gavin McDonald <gm...@apache.org>
> wrote:
>
>> There are MANY JDKS installed already. Oracle JDKs OpenJDKs AdoptOpenJDKs
>> - they are all already there.
>>
>> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>>
>> On Mon, Jul 26, 2021 at 11:47 AM Arnaud Héritier <ah...@gmail.com>
>> wrote:
>>
>> > It has to be discussed with infra.
>> > I am not sure which distro is used.
>> > It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
>> > something else than Java 8 - no comment)
>> > I don't have the feeling for now (with the data I reviewed) that it's a
>> > memory / GC issue (it could create disconnections under high load)
>> > Here the stability issue occurs even when the controller does nothing
>> (the
>> > controller cannot ping the agent or vice and versa) and it seems to
>> impact
>> > more the linux agents than the windows ones (it's a pity)
>> >
>> >
>> >
>> > On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <ti...@apache.org>
>> > wrote:
>> >
>> >> Can you install AdoptOpenJdk for the Jenkins controller?
>> >> It contains Eclipse OpenJ9 Garbage Collector and it significantly
>> >> decreases
>> >> memory consumption of the application due to the meta space goes to the
>> >> disk.
>> >> You should save 40 - 75% out of 3GB.
>> >> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most
>> memory.
>> >>
>> >> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <ah...@gmail.com>
>> >> wrote:
>> >>
>> >> > yes for the controller it depends of its size (number of jobs and
>> types
>> >> of
>> >> > jobs) but here we are fine it seems with our 3Gb
>> >> >
>> >> > * Java
>> >> > - Version: 1.8.0_292
>> >> > - Maximum memory: 3.00 GB (3221225472)
>> >> > - Allocated memory: 3.00 GB (3221225472)
>> >> > - Free memory: 750.15 MB (786591664)
>> >> > - In-use memory: 2.27 GB (2434633808)
>> >> > - GC strategy: G1
>> >> > - Available CPUs: 2
>> >> >
>> >> > For agents I reduced the memory allocated to the agent process but it
>> >> > doesn't help much (it seems - even if it is still a good thing to do)
>> >> >
>> >> > What is strange is that I see our agents sometimes disconnected even
>> >> when
>> >> > we have no activity on the jenkins controller
>> >> >
>> >> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get access
>> to
>> >> its
>> >> > logs
>> >> >
>> >> > In general the connection lost is detected by what we call the
>> >> PingThread (
>> >> >
>> >> >
>> >>
>> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
>> >> > ) but not only
>> >> >
>> >> > https://ci-maven.apache.org/log/all
>> >> >
>> >> > For example it was few minutes ago we got 3 agents disconnected while
>> >> > nothing was running
>> >> >
>> >> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
>> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> > Ping failed. Terminating the channel maven4.
>> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861769
>> >> hasn't
>> >> > completed by 1626937101769
>> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
>> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> > Ping failed. Terminating the channel maven3.
>> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861777
>> >> hasn't
>> >> > completed by 1626937101778
>> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
>> >> > hudson.slaves.ChannelPinger$1#onDead:
>> >> > Ping failed. Terminating the channel maven5.
>> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861982
>> >> hasn't
>> >> > completed by 1626937101983
>> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >> >
>> >> > @Gavin McDonald <gm...@apache.org> In terms of network, is it
>> the
>> >> same
>> >> > environment we use today compared to the ci-builds.apache.org
>> >> environment
>> >> > ?
>> >> >
>> >> >
>> >> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <
>> tibordigana@apache.org>
>> >> > wrote:
>> >> >
>> >> > > In my company, I also used 1GB for Xmx of Java Heap for the Jenkins
>> >> JVM
>> >> > and
>> >> > > it was enough.
>> >> > > The subprocesses like Maven need to have much more memory to
>> allocate
>> >> for
>> >> > > themself rather than Jenkins JVM.
>> >> > > T
>> >> > >
>> >> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <
>> aheritier@gmail.com>
>> >> > > wrote:
>> >> > >
>> >> > > > I am looking at our builds and I try to understand why our agents
>> >> are
>> >> > > often
>> >> > > > disconnected during the builds.
>> >> > > > We have in general a stacktrace like
>> >> > > >
>> >> > > > maven6 was marked offline: Connection was broken:
>> >> java.io.IOException:
>> >> > > > Pipe closed after 0 cycles
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
>> >> > > > at
>> >> > > >
>> >> >
>> >>
>> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
>> >> > > > at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > As far I can see we are using 16Gb "hosts" for linux agents
>> >> > > >
>> >> > > > Something very strange is that the jenkins agent (this small
>> >> component
>> >> > > > doing the link between the build host and the controller) is
>> >> configured
>> >> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the
>> server
>> >> mem
>> >> > > > (even more because of the non-heap)
>> >> > > > This one in general should require in general really less. 1Gb is
>> >> > > already a
>> >> > > > lot from my exp.
>> >> > > > Due to this, the OS can see it has the biggest process on the
>> host
>> >> and
>> >> > > > decide to kill it when the rest of the memory is used by the
>> build.
>> >> > > > I think we should decrease this value.
>> >> > > > (I can do it but I don't know how was configured the
>> ci.apache.org
>> >> > > agents
>> >> > > > and I would like to not add more issue if this setting was here
>> in
>> >> the
>> >> > > past
>> >> > > >
>> >> > > > I don't think it is the root cause of our instabilities (at least
>> >> all)
>> >> > > and
>> >> > > > there is something else I have to find but it's a cheap fix to
>> try
>> >> > > >
>> >> > > > FYI our agents VMs are ~like this today:
>> >> > > >
>> >> > > > - Java
>> >> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
>> >> > > > + Vendor: Oracle Corporation
>> >> > > > + Version: 1.8.0_291
>> >> > > > + Maximum memory: 7.67 GB (8232370176)
>> >> > > > + Allocated memory: 7.67 GB (8232370176)
>> >> > > > + Free memory: 6.03 GB (6470953760)
>> >> > > > + In-use memory: 1.64 GB (1761416416)
>> >> > > > + GC strategy: ParallelGC
>> >> > > > + Available CPUs: 4
>> >> > > >
>> >> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as the
>> >> Free
>> >> > mem
>> >> > > > is high)
>> >> > > >
>> >> > > > I would be in favor to try to launch them with -Xms128m
>> >> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>> >> > > >
>> >> > > > I think it's enough customization to start with
>> >> > > >
>> >> > > > Cheers
>> >> > > >
>> >> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <
>> >> aheritier@gmail.com>
>> >> > > > wrote:
>> >> > > >
>> >> > > > > I am not sure about the setup
>> >> > > > > AFAICS we don't use any JDK installer (
>> >> > > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose
>> that
>> >> > the
>> >> > > > > different JDKs are supposed to be installed directly on the
>> agent
>> >> ?
>> >> > > > > I am not sure how it was done on the previous environment
>> >> > > > >
>> >> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
>> >> tibordigana@apache.org
>> >> > >
>> >> > > > > wrote:
>> >> > > > >
>> >> > > > >> The new CI system has the following issue:
>> >> > > > >>
>> >> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
>> >> > > > >>
>> >> > > > >>
>> >> > > > >>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>> >> > > > >>
>> >> > > > >>
>> >> > > > >>
>> >> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
>> >> > gmcdonald@apache.org>
>> >> > > > >> wrote:
>> >> > > > >>
>> >> > > > >> > Hi Maven folks.
>> >> > > > >> >
>> >> > > > >> > Infra has decided to separate off the Maven build jobs from
>> >> > > > >> > ci-builds.apache.org over to its very own Jenkins
>> Controller
>> >> and
>> >> > > > >> Agents.
>> >> > > > >> >
>> >> > > > >> > This means that Maven now has a dedicated Jenkins
>> environment
>> >> for
>> >> > > > >> itself.
>> >> > > > >> > It
>> >> > > > >> > also means that no other projects build jobs can build on
>> the
>> >> > Maven
>> >> > > > >> nodes;
>> >> > > > >> > and
>> >> > > > >> > then Maven jobs will no longer be able to build on the
>> >> ci-builds
>> >> > > > jobs.
>> >> > > > >> >
>> >> > > > >> > Your new Controller is set up as
>> https://ci-maven.apache.org
>> >> and
>> >> > > all
>> >> > > > >> Maven
>> >> > > > >> > Committers
>> >> > > > >> > can login via LDAP and create jobs.
>> >> > > > >> >
>> >> > > > >> > At the time of writing, there is one node/agent attached
>> but I
>> >> am
>> >> > > > >> building
>> >> > > > >> > 4 more - all
>> >> > > > >> > Ubuntu 20.04 and based in our Azure account.
>> >> > > > >> >
>> >> > > > >> > We can automagically move all your jobs over from ci-builds
>> to
>> >> > > > ci-maven
>> >> > > > >> - I
>> >> > > > >> > just need someone to tell me go ahead and do it.
>> >> > > > >> >
>> >> > > > >> > In the meantime, feel free to have a test. The remaining 4
>> >> agents
>> >> > > will
>> >> > > > >> be
>> >> > > > >> > online
>> >> > > > >> > by tomorrow. We will review after a month if 5 is enough
>> nodes.
>> >> > > > >> >
>> >> > > > >> > As with other projects having their own dedicated
>> controller,
>> >> who
>> >> > > have
>> >> > > > >> > taken advantage
>> >> > > > >> > of this isolation by having some nodes/agents given to the
>> >> project
>> >> > > as
>> >> > > > a
>> >> > > > >> > 'targeted donation'
>> >> > > > >> > so someone here may know of a Company will to donate 5 - 10
>> or
>> >> > more
>> >> > > > >> nodes
>> >> > > > >> > specifically
>> >> > > > >> > for Maven Jenkins environment. Infra can afford to hand you
>> >> over 5
>> >> > > > right
>> >> > > > >> > now.
>> >> > > > >> >
>> >> > > > >> > Let me know if you have any questions, otherwise let me know
>> >> when
>> >> > I
>> >> > > > can
>> >> > > > >> > make the
>> >> > > > >> > transfer of your jobs.
>> >> > > > >> >
>> >> > > > >> > Thanks
>> >> > > > >> >
>> >> > > > >> > --
>> >> > > > >> >
>> >> > > > >> > *Gavin McDonald*
>> >> > > > >> > Systems Administrator
>> >> > > > >> > ASF Infrastructure Team
>> >> > > > >> >
>> >> > > > >>
>> >> > > > >
>> >> > > > >
>> >> > > > > --
>> >> > > > > Arnaud Héritier
>> >> > > > > Twitter/Skype : aheritier
>> >> > > > >
>> >> > > >
>> >> > > >
>> >> > > > --
>> >> > > > Arnaud Héritier
>> >> > > > Twitter/Skype : aheritier
>> >> > > >
>> >> > >
>> >> >
>> >> >
>> >> > --
>> >> > Arnaud Héritier
>> >> > Twitter/Skype : aheritier
>> >> >
>> >>
>> >
>> >
>> > --
>> > Arnaud Héritier
>> > Twitter/Skype : aheritier
>> >
>>
>>
>> --
>>
>> *Gavin McDonald*
>> Systems Administrator
>> ASF Infrastructure Team
>>
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
Gavin, these JDKs are only for build agents, right ?
Tibor was asking for the JVM used to host Tomcat/Jenkins.
(And I suppose that the controller part is templatised)
On Mon, Jul 26, 2021 at 1:06 PM Gavin McDonald <gm...@apache.org> wrote:
> There are MANY JDKS installed already. Oracle JDKs OpenJDKs AdoptOpenJDKs
> - they are all already there.
>
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>
> On Mon, Jul 26, 2021 at 11:47 AM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
> > It has to be discussed with infra.
> > I am not sure which distro is used.
> > It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
> > something else than Java 8 - no comment)
> > I don't have the feeling for now (with the data I reviewed) that it's a
> > memory / GC issue (it could create disconnections under high load)
> > Here the stability issue occurs even when the controller does nothing
> (the
> > controller cannot ping the agent or vice and versa) and it seems to
> impact
> > more the linux agents than the windows ones (it's a pity)
> >
> >
> >
> > On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <ti...@apache.org>
> > wrote:
> >
> >> Can you install AdoptOpenJdk for the Jenkins controller?
> >> It contains Eclipse OpenJ9 Garbage Collector and it significantly
> >> decreases
> >> memory consumption of the application due to the meta space goes to the
> >> disk.
> >> You should save 40 - 75% out of 3GB.
> >> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most
> memory.
> >>
> >> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <ah...@gmail.com>
> >> wrote:
> >>
> >> > yes for the controller it depends of its size (number of jobs and
> types
> >> of
> >> > jobs) but here we are fine it seems with our 3Gb
> >> >
> >> > * Java
> >> > - Version: 1.8.0_292
> >> > - Maximum memory: 3.00 GB (3221225472)
> >> > - Allocated memory: 3.00 GB (3221225472)
> >> > - Free memory: 750.15 MB (786591664)
> >> > - In-use memory: 2.27 GB (2434633808)
> >> > - GC strategy: G1
> >> > - Available CPUs: 2
> >> >
> >> > For agents I reduced the memory allocated to the agent process but it
> >> > doesn't help much (it seems - even if it is still a good thing to do)
> >> >
> >> > What is strange is that I see our agents sometimes disconnected even
> >> when
> >> > we have no activity on the jenkins controller
> >> >
> >> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get access to
> >> its
> >> > logs
> >> >
> >> > In general the connection lost is detected by what we call the
> >> PingThread (
> >> >
> >> >
> >>
> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
> >> > ) but not only
> >> >
> >> > https://ci-maven.apache.org/log/all
> >> >
> >> > For example it was few minutes ago we got 3 agents disconnected while
> >> > nothing was running
> >> >
> >> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> > Ping failed. Terminating the channel maven4.
> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861769
> >> hasn't
> >> > completed by 1626937101769
> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> > Ping failed. Terminating the channel maven3.
> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861777
> >> hasn't
> >> > completed by 1626937101778
> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
> >> > hudson.slaves.ChannelPinger$1#onDead:
> >> > Ping failed. Terminating the channel maven5.
> >> > java.util.concurrent.TimeoutException: Ping started at 1626936861982
> >> hasn't
> >> > completed by 1626937101983
> >> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> >> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >> >
> >> > @Gavin McDonald <gm...@apache.org> In terms of network, is it the
> >> same
> >> > environment we use today compared to the ci-builds.apache.org
> >> environment
> >> > ?
> >> >
> >> >
> >> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <tibordigana@apache.org
> >
> >> > wrote:
> >> >
> >> > > In my company, I also used 1GB for Xmx of Java Heap for the Jenkins
> >> JVM
> >> > and
> >> > > it was enough.
> >> > > The subprocesses like Maven need to have much more memory to
> allocate
> >> for
> >> > > themself rather than Jenkins JVM.
> >> > > T
> >> > >
> >> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <
> aheritier@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > I am looking at our builds and I try to understand why our agents
> >> are
> >> > > often
> >> > > > disconnected during the builds.
> >> > > > We have in general a stacktrace like
> >> > > >
> >> > > > maven6 was marked offline: Connection was broken:
> >> java.io.IOException:
> >> > > > Pipe closed after 0 cycles
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> >> > > > at
> >> > > >
> >> >
> >>
> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> >> > > > at
> >> > > >
> >> > >
> >> >
> >>
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
> >> > > >
> >> > > >
> >> > > >
> >> > > > As far I can see we are using 16Gb "hosts" for linux agents
> >> > > >
> >> > > > Something very strange is that the jenkins agent (this small
> >> component
> >> > > > doing the link between the build host and the controller) is
> >> configured
> >> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the
> server
> >> mem
> >> > > > (even more because of the non-heap)
> >> > > > This one in general should require in general really less. 1Gb is
> >> > > already a
> >> > > > lot from my exp.
> >> > > > Due to this, the OS can see it has the biggest process on the host
> >> and
> >> > > > decide to kill it when the rest of the memory is used by the
> build.
> >> > > > I think we should decrease this value.
> >> > > > (I can do it but I don't know how was configured the
> ci.apache.org
> >> > > agents
> >> > > > and I would like to not add more issue if this setting was here in
> >> the
> >> > > past
> >> > > >
> >> > > > I don't think it is the root cause of our instabilities (at least
> >> all)
> >> > > and
> >> > > > there is something else I have to find but it's a cheap fix to try
> >> > > >
> >> > > > FYI our agents VMs are ~like this today:
> >> > > >
> >> > > > - Java
> >> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> >> > > > + Vendor: Oracle Corporation
> >> > > > + Version: 1.8.0_291
> >> > > > + Maximum memory: 7.67 GB (8232370176)
> >> > > > + Allocated memory: 7.67 GB (8232370176)
> >> > > > + Free memory: 6.03 GB (6470953760)
> >> > > > + In-use memory: 1.64 GB (1761416416)
> >> > > > + GC strategy: ParallelGC
> >> > > > + Available CPUs: 4
> >> > > >
> >> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as the
> >> Free
> >> > mem
> >> > > > is high)
> >> > > >
> >> > > > I would be in favor to try to launch them with -Xms128m
> >> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
> >> > > >
> >> > > > I think it's enough customization to start with
> >> > > >
> >> > > > Cheers
> >> > > >
> >> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <
> >> aheritier@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > I am not sure about the setup
> >> > > > > AFAICS we don't use any JDK installer (
> >> > > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose
> that
> >> > the
> >> > > > > different JDKs are supposed to be installed directly on the
> agent
> >> ?
> >> > > > > I am not sure how it was done on the previous environment
> >> > > > >
> >> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
> >> tibordigana@apache.org
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> The new CI system has the following issue:
> >> > > > >>
> >> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
> >> > gmcdonald@apache.org>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > Hi Maven folks.
> >> > > > >> >
> >> > > > >> > Infra has decided to separate off the Maven build jobs from
> >> > > > >> > ci-builds.apache.org over to its very own Jenkins Controller
> >> and
> >> > > > >> Agents.
> >> > > > >> >
> >> > > > >> > This means that Maven now has a dedicated Jenkins environment
> >> for
> >> > > > >> itself.
> >> > > > >> > It
> >> > > > >> > also means that no other projects build jobs can build on the
> >> > Maven
> >> > > > >> nodes;
> >> > > > >> > and
> >> > > > >> > then Maven jobs will no longer be able to build on the
> >> ci-builds
> >> > > > jobs.
> >> > > > >> >
> >> > > > >> > Your new Controller is set up as https://ci-maven.apache.org
> >> and
> >> > > all
> >> > > > >> Maven
> >> > > > >> > Committers
> >> > > > >> > can login via LDAP and create jobs.
> >> > > > >> >
> >> > > > >> > At the time of writing, there is one node/agent attached but
> I
> >> am
> >> > > > >> building
> >> > > > >> > 4 more - all
> >> > > > >> > Ubuntu 20.04 and based in our Azure account.
> >> > > > >> >
> >> > > > >> > We can automagically move all your jobs over from ci-builds
> to
> >> > > > ci-maven
> >> > > > >> - I
> >> > > > >> > just need someone to tell me go ahead and do it.
> >> > > > >> >
> >> > > > >> > In the meantime, feel free to have a test. The remaining 4
> >> agents
> >> > > will
> >> > > > >> be
> >> > > > >> > online
> >> > > > >> > by tomorrow. We will review after a month if 5 is enough
> nodes.
> >> > > > >> >
> >> > > > >> > As with other projects having their own dedicated controller,
> >> who
> >> > > have
> >> > > > >> > taken advantage
> >> > > > >> > of this isolation by having some nodes/agents given to the
> >> project
> >> > > as
> >> > > > a
> >> > > > >> > 'targeted donation'
> >> > > > >> > so someone here may know of a Company will to donate 5 - 10
> or
> >> > more
> >> > > > >> nodes
> >> > > > >> > specifically
> >> > > > >> > for Maven Jenkins environment. Infra can afford to hand you
> >> over 5
> >> > > > right
> >> > > > >> > now.
> >> > > > >> >
> >> > > > >> > Let me know if you have any questions, otherwise let me know
> >> when
> >> > I
> >> > > > can
> >> > > > >> > make the
> >> > > > >> > transfer of your jobs.
> >> > > > >> >
> >> > > > >> > Thanks
> >> > > > >> >
> >> > > > >> > --
> >> > > > >> >
> >> > > > >> > *Gavin McDonald*
> >> > > > >> > Systems Administrator
> >> > > > >> > ASF Infrastructure Team
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Arnaud Héritier
> >> > > > > Twitter/Skype : aheritier
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Arnaud Héritier
> >> > > > Twitter/Skype : aheritier
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > Arnaud Héritier
> >> > Twitter/Skype : aheritier
> >> >
> >>
> >
> >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
There are MANY JDKS installed already. Oracle JDKs OpenJDKs AdoptOpenJDKs
- they are all already there.
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
On Mon, Jul 26, 2021 at 11:47 AM Arnaud Héritier <ah...@gmail.com>
wrote:
> It has to be discussed with infra.
> I am not sure which distro is used.
> It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
> something else than Java 8 - no comment)
> I don't have the feeling for now (with the data I reviewed) that it's a
> memory / GC issue (it could create disconnections under high load)
> Here the stability issue occurs even when the controller does nothing (the
> controller cannot ping the agent or vice and versa) and it seems to impact
> more the linux agents than the windows ones (it's a pity)
>
>
>
> On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <ti...@apache.org>
> wrote:
>
>> Can you install AdoptOpenJdk for the Jenkins controller?
>> It contains Eclipse OpenJ9 Garbage Collector and it significantly
>> decreases
>> memory consumption of the application due to the meta space goes to the
>> disk.
>> You should save 40 - 75% out of 3GB.
>> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most memory.
>>
>> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <ah...@gmail.com>
>> wrote:
>>
>> > yes for the controller it depends of its size (number of jobs and types
>> of
>> > jobs) but here we are fine it seems with our 3Gb
>> >
>> > * Java
>> > - Version: 1.8.0_292
>> > - Maximum memory: 3.00 GB (3221225472)
>> > - Allocated memory: 3.00 GB (3221225472)
>> > - Free memory: 750.15 MB (786591664)
>> > - In-use memory: 2.27 GB (2434633808)
>> > - GC strategy: G1
>> > - Available CPUs: 2
>> >
>> > For agents I reduced the memory allocated to the agent process but it
>> > doesn't help much (it seems - even if it is still a good thing to do)
>> >
>> > What is strange is that I see our agents sometimes disconnected even
>> when
>> > we have no activity on the jenkins controller
>> >
>> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get access to
>> its
>> > logs
>> >
>> > In general the connection lost is detected by what we call the
>> PingThread (
>> >
>> >
>> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
>> > ) but not only
>> >
>> > https://ci-maven.apache.org/log/all
>> >
>> > For example it was few minutes ago we got 3 agents disconnected while
>> > nothing was running
>> >
>> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
>> > hudson.slaves.ChannelPinger$1#onDead:
>> > Ping failed. Terminating the channel maven4.
>> > java.util.concurrent.TimeoutException: Ping started at 1626936861769
>> hasn't
>> > completed by 1626937101769
>> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
>> > hudson.slaves.ChannelPinger$1#onDead:
>> > Ping failed. Terminating the channel maven3.
>> > java.util.concurrent.TimeoutException: Ping started at 1626936861777
>> hasn't
>> > completed by 1626937101778
>> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
>> > hudson.slaves.ChannelPinger$1#onDead:
>> > Ping failed. Terminating the channel maven5.
>> > java.util.concurrent.TimeoutException: Ping started at 1626936861982
>> hasn't
>> > completed by 1626937101983
>> > at hudson.remoting.PingThread.ping(PingThread.java:134)
>> > at hudson.remoting.PingThread.run(PingThread.java:90)
>> >
>> > @Gavin McDonald <gm...@apache.org> In terms of network, is it the
>> same
>> > environment we use today compared to the ci-builds.apache.org
>> environment
>> > ?
>> >
>> >
>> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <ti...@apache.org>
>> > wrote:
>> >
>> > > In my company, I also used 1GB for Xmx of Java Heap for the Jenkins
>> JVM
>> > and
>> > > it was enough.
>> > > The subprocesses like Maven need to have much more memory to allocate
>> for
>> > > themself rather than Jenkins JVM.
>> > > T
>> > >
>> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com>
>> > > wrote:
>> > >
>> > > > I am looking at our builds and I try to understand why our agents
>> are
>> > > often
>> > > > disconnected during the builds.
>> > > > We have in general a stacktrace like
>> > > >
>> > > > maven6 was marked offline: Connection was broken:
>> java.io.IOException:
>> > > > Pipe closed after 0 cycles
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
>> > > > at
>> > > >
>> > >
>> >
>> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
>> > > > at
>> > > >
>> >
>> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
>> > > > at
>> > > >
>> > >
>> >
>> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
>> > > > at
>> > > >
>> > >
>> >
>> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
>> > > > at
>> > > >
>> > >
>> >
>> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
>> > > > at
>> > > >
>> > >
>> >
>> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>> > > >
>> > > >
>> > > >
>> > > > As far I can see we are using 16Gb "hosts" for linux agents
>> > > >
>> > > > Something very strange is that the jenkins agent (this small
>> component
>> > > > doing the link between the build host and the controller) is
>> configured
>> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server
>> mem
>> > > > (even more because of the non-heap)
>> > > > This one in general should require in general really less. 1Gb is
>> > > already a
>> > > > lot from my exp.
>> > > > Due to this, the OS can see it has the biggest process on the host
>> and
>> > > > decide to kill it when the rest of the memory is used by the build.
>> > > > I think we should decrease this value.
>> > > > (I can do it but I don't know how was configured the ci.apache.org
>> > > agents
>> > > > and I would like to not add more issue if this setting was here in
>> the
>> > > past
>> > > >
>> > > > I don't think it is the root cause of our instabilities (at least
>> all)
>> > > and
>> > > > there is something else I have to find but it's a cheap fix to try
>> > > >
>> > > > FYI our agents VMs are ~like this today:
>> > > >
>> > > > - Java
>> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
>> > > > + Vendor: Oracle Corporation
>> > > > + Version: 1.8.0_291
>> > > > + Maximum memory: 7.67 GB (8232370176)
>> > > > + Allocated memory: 7.67 GB (8232370176)
>> > > > + Free memory: 6.03 GB (6470953760)
>> > > > + In-use memory: 1.64 GB (1761416416)
>> > > > + GC strategy: ParallelGC
>> > > > + Available CPUs: 4
>> > > >
>> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as the
>> Free
>> > mem
>> > > > is high)
>> > > >
>> > > > I would be in favor to try to launch them with -Xms128m
>> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>> > > >
>> > > > I think it's enough customization to start with
>> > > >
>> > > > Cheers
>> > > >
>> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <
>> aheritier@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I am not sure about the setup
>> > > > > AFAICS we don't use any JDK installer (
>> > > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose that
>> > the
>> > > > > different JDKs are supposed to be installed directly on the agent
>> ?
>> > > > > I am not sure how it was done on the previous environment
>> > > > >
>> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
>> tibordigana@apache.org
>> > >
>> > > > > wrote:
>> > > > >
>> > > > >> The new CI system has the following issue:
>> > > > >>
>> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
>> > > > >>
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
>> > gmcdonald@apache.org>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hi Maven folks.
>> > > > >> >
>> > > > >> > Infra has decided to separate off the Maven build jobs from
>> > > > >> > ci-builds.apache.org over to its very own Jenkins Controller
>> and
>> > > > >> Agents.
>> > > > >> >
>> > > > >> > This means that Maven now has a dedicated Jenkins environment
>> for
>> > > > >> itself.
>> > > > >> > It
>> > > > >> > also means that no other projects build jobs can build on the
>> > Maven
>> > > > >> nodes;
>> > > > >> > and
>> > > > >> > then Maven jobs will no longer be able to build on the
>> ci-builds
>> > > > jobs.
>> > > > >> >
>> > > > >> > Your new Controller is set up as https://ci-maven.apache.org
>> and
>> > > all
>> > > > >> Maven
>> > > > >> > Committers
>> > > > >> > can login via LDAP and create jobs.
>> > > > >> >
>> > > > >> > At the time of writing, there is one node/agent attached but I
>> am
>> > > > >> building
>> > > > >> > 4 more - all
>> > > > >> > Ubuntu 20.04 and based in our Azure account.
>> > > > >> >
>> > > > >> > We can automagically move all your jobs over from ci-builds to
>> > > > ci-maven
>> > > > >> - I
>> > > > >> > just need someone to tell me go ahead and do it.
>> > > > >> >
>> > > > >> > In the meantime, feel free to have a test. The remaining 4
>> agents
>> > > will
>> > > > >> be
>> > > > >> > online
>> > > > >> > by tomorrow. We will review after a month if 5 is enough nodes.
>> > > > >> >
>> > > > >> > As with other projects having their own dedicated controller,
>> who
>> > > have
>> > > > >> > taken advantage
>> > > > >> > of this isolation by having some nodes/agents given to the
>> project
>> > > as
>> > > > a
>> > > > >> > 'targeted donation'
>> > > > >> > so someone here may know of a Company will to donate 5 - 10 or
>> > more
>> > > > >> nodes
>> > > > >> > specifically
>> > > > >> > for Maven Jenkins environment. Infra can afford to hand you
>> over 5
>> > > > right
>> > > > >> > now.
>> > > > >> >
>> > > > >> > Let me know if you have any questions, otherwise let me know
>> when
>> > I
>> > > > can
>> > > > >> > make the
>> > > > >> > transfer of your jobs.
>> > > > >> >
>> > > > >> > Thanks
>> > > > >> >
>> > > > >> > --
>> > > > >> >
>> > > > >> > *Gavin McDonald*
>> > > > >> > Systems Administrator
>> > > > >> > ASF Infrastructure Team
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Arnaud Héritier
>> > > > > Twitter/Skype : aheritier
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Arnaud Héritier
>> > > > Twitter/Skype : aheritier
>> > > >
>> > >
>> >
>> >
>> > --
>> > Arnaud Héritier
>> > Twitter/Skype : aheritier
>> >
>>
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
It has to be discussed with infra.
I am not sure which distro is used.
It's a Private Build of openJDK 8 (today CloudBees CI doesn't support
something else than Java 8 - no comment)
I don't have the feeling for now (with the data I reviewed) that it's a
memory / GC issue (it could create disconnections under high load)
Here the stability issue occurs even when the controller does nothing (the
controller cannot ping the agent or vice and versa) and it seems to impact
more the linux agents than the windows ones (it's a pity)
On Fri, Jul 23, 2021 at 12:06 AM Tibor Digana <ti...@apache.org>
wrote:
> Can you install AdoptOpenJdk for the Jenkins controller?
> It contains Eclipse OpenJ9 Garbage Collector and it significantly decreases
> memory consumption of the application due to the meta space goes to the
> disk.
> You should save 40 - 75% out of 3GB.
> I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most memory.
>
> On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
> > yes for the controller it depends of its size (number of jobs and types
> of
> > jobs) but here we are fine it seems with our 3Gb
> >
> > * Java
> > - Version: 1.8.0_292
> > - Maximum memory: 3.00 GB (3221225472)
> > - Allocated memory: 3.00 GB (3221225472)
> > - Free memory: 750.15 MB (786591664)
> > - In-use memory: 2.27 GB (2434633808)
> > - GC strategy: G1
> > - Available CPUs: 2
> >
> > For agents I reduced the memory allocated to the agent process but it
> > doesn't help much (it seems - even if it is still a good thing to do)
> >
> > What is strange is that I see our agents sometimes disconnected even when
> > we have no activity on the jenkins controller
> >
> > Sadly jenkins is deployed on Apache Tomcat thus I cannot get access to
> its
> > logs
> >
> > In general the connection lost is detected by what we call the
> PingThread (
> >
> >
> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
> > ) but not only
> >
> > https://ci-maven.apache.org/log/all
> >
> > For example it was few minutes ago we got 3 agents disconnected while
> > nothing was running
> >
> > 2021-07-22 06:58:21.769+0000 [id=106291] INFO
> > hudson.slaves.ChannelPinger$1#onDead:
> > Ping failed. Terminating the channel maven4.
> > java.util.concurrent.TimeoutException: Ping started at 1626936861769
> hasn't
> > completed by 1626937101769
> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> > at hudson.remoting.PingThread.run(PingThread.java:90)
> > 2021-07-22 06:58:21.778+0000 [id=106292] INFO
> > hudson.slaves.ChannelPinger$1#onDead:
> > Ping failed. Terminating the channel maven3.
> > java.util.concurrent.TimeoutException: Ping started at 1626936861777
> hasn't
> > completed by 1626937101778
> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> > at hudson.remoting.PingThread.run(PingThread.java:90)
> > 2021-07-22 06:58:21.983+0000 [id=106295] INFO
> > hudson.slaves.ChannelPinger$1#onDead:
> > Ping failed. Terminating the channel maven5.
> > java.util.concurrent.TimeoutException: Ping started at 1626936861982
> hasn't
> > completed by 1626937101983
> > at hudson.remoting.PingThread.ping(PingThread.java:134)
> > at hudson.remoting.PingThread.run(PingThread.java:90)
> >
> > @Gavin McDonald <gm...@apache.org> In terms of network, is it the
> same
> > environment we use today compared to the ci-builds.apache.org
> environment
> > ?
> >
> >
> > On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <ti...@apache.org>
> > wrote:
> >
> > > In my company, I also used 1GB for Xmx of Java Heap for the Jenkins JVM
> > and
> > > it was enough.
> > > The subprocesses like Maven need to have much more memory to allocate
> for
> > > themself rather than Jenkins JVM.
> > > T
> > >
> > > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com>
> > > wrote:
> > >
> > > > I am looking at our builds and I try to understand why our agents are
> > > often
> > > > disconnected during the builds.
> > > > We have in general a stacktrace like
> > > >
> > > > maven6 was marked offline: Connection was broken:
> java.io.IOException:
> > > > Pipe closed after 0 cycles
> > > > at
> > > >
> > >
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> > > > at
> > > >
> > >
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> > > > at
> > > >
> > >
> >
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> > > > at
> > > >
> > hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> > > > at
> > > >
> > >
> >
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> > > > at
> > > >
> > >
> >
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> > > > at
> > > >
> > >
> >
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> > > > at
> > > >
> > >
> >
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
> > > >
> > > >
> > > >
> > > > As far I can see we are using 16Gb "hosts" for linux agents
> > > >
> > > > Something very strange is that the jenkins agent (this small
> component
> > > > doing the link between the build host and the controller) is
> configured
> > > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server
> mem
> > > > (even more because of the non-heap)
> > > > This one in general should require in general really less. 1Gb is
> > > already a
> > > > lot from my exp.
> > > > Due to this, the OS can see it has the biggest process on the host
> and
> > > > decide to kill it when the rest of the memory is used by the build.
> > > > I think we should decrease this value.
> > > > (I can do it but I don't know how was configured the ci.apache.org
> > > agents
> > > > and I would like to not add more issue if this setting was here in
> the
> > > past
> > > >
> > > > I don't think it is the root cause of our instabilities (at least
> all)
> > > and
> > > > there is something else I have to find but it's a cheap fix to try
> > > >
> > > > FYI our agents VMs are ~like this today:
> > > >
> > > > - Java
> > > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> > > > + Vendor: Oracle Corporation
> > > > + Version: 1.8.0_291
> > > > + Maximum memory: 7.67 GB (8232370176)
> > > > + Allocated memory: 7.67 GB (8232370176)
> > > > + Free memory: 6.03 GB (6470953760)
> > > > + In-use memory: 1.64 GB (1761416416)
> > > > + GC strategy: ParallelGC
> > > > + Available CPUs: 4
> > > >
> > > > 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free
> > mem
> > > > is high)
> > > >
> > > > I would be in favor to try to launch them with -Xms128m
> > > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
> > > >
> > > > I think it's enough customization to start with
> > > >
> > > > Cheers
> > > >
> > > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <aheritier@gmail.com
> >
> > > > wrote:
> > > >
> > > > > I am not sure about the setup
> > > > > AFAICS we don't use any JDK installer (
> > > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose that
> > the
> > > > > different JDKs are supposed to be installed directly on the agent ?
> > > > > I am not sure how it was done on the previous environment
> > > > >
> > > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <
> tibordigana@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > >> The new CI system has the following issue:
> > > > >>
> > > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
> > gmcdonald@apache.org>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Maven folks.
> > > > >> >
> > > > >> > Infra has decided to separate off the Maven build jobs from
> > > > >> > ci-builds.apache.org over to its very own Jenkins Controller
> and
> > > > >> Agents.
> > > > >> >
> > > > >> > This means that Maven now has a dedicated Jenkins environment
> for
> > > > >> itself.
> > > > >> > It
> > > > >> > also means that no other projects build jobs can build on the
> > Maven
> > > > >> nodes;
> > > > >> > and
> > > > >> > then Maven jobs will no longer be able to build on the
> ci-builds
> > > > jobs.
> > > > >> >
> > > > >> > Your new Controller is set up as https://ci-maven.apache.org
> and
> > > all
> > > > >> Maven
> > > > >> > Committers
> > > > >> > can login via LDAP and create jobs.
> > > > >> >
> > > > >> > At the time of writing, there is one node/agent attached but I
> am
> > > > >> building
> > > > >> > 4 more - all
> > > > >> > Ubuntu 20.04 and based in our Azure account.
> > > > >> >
> > > > >> > We can automagically move all your jobs over from ci-builds to
> > > > ci-maven
> > > > >> - I
> > > > >> > just need someone to tell me go ahead and do it.
> > > > >> >
> > > > >> > In the meantime, feel free to have a test. The remaining 4
> agents
> > > will
> > > > >> be
> > > > >> > online
> > > > >> > by tomorrow. We will review after a month if 5 is enough nodes.
> > > > >> >
> > > > >> > As with other projects having their own dedicated controller,
> who
> > > have
> > > > >> > taken advantage
> > > > >> > of this isolation by having some nodes/agents given to the
> project
> > > as
> > > > a
> > > > >> > 'targeted donation'
> > > > >> > so someone here may know of a Company will to donate 5 - 10 or
> > more
> > > > >> nodes
> > > > >> > specifically
> > > > >> > for Maven Jenkins environment. Infra can afford to hand you
> over 5
> > > > right
> > > > >> > now.
> > > > >> >
> > > > >> > Let me know if you have any questions, otherwise let me know
> when
> > I
> > > > can
> > > > >> > make the
> > > > >> > transfer of your jobs.
> > > > >> >
> > > > >> > Thanks
> > > > >> >
> > > > >> > --
> > > > >> >
> > > > >> > *Gavin McDonald*
> > > > >> > Systems Administrator
> > > > >> > ASF Infrastructure Team
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Arnaud Héritier
> > > > > Twitter/Skype : aheritier
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnaud Héritier
> > > > Twitter/Skype : aheritier
> > > >
> > >
> >
> >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Tibor Digana <ti...@apache.org>.
Can you install AdoptOpenJdk for the Jenkins controller?
It contains Eclipse OpenJ9 Garbage Collector and it significantly decreases
memory consumption of the application due to the meta space goes to the
disk.
You should save 40 - 75% out of 3GB.
I used G1, Shenandoah, ZGC and Eclipse OpenJ9 which saved the most memory.
On Thu, Jul 22, 2021 at 9:23 AM Arnaud Héritier <ah...@gmail.com> wrote:
> yes for the controller it depends of its size (number of jobs and types of
> jobs) but here we are fine it seems with our 3Gb
>
> * Java
> - Version: 1.8.0_292
> - Maximum memory: 3.00 GB (3221225472)
> - Allocated memory: 3.00 GB (3221225472)
> - Free memory: 750.15 MB (786591664)
> - In-use memory: 2.27 GB (2434633808)
> - GC strategy: G1
> - Available CPUs: 2
>
> For agents I reduced the memory allocated to the agent process but it
> doesn't help much (it seems - even if it is still a good thing to do)
>
> What is strange is that I see our agents sometimes disconnected even when
> we have no activity on the jenkins controller
>
> Sadly jenkins is deployed on Apache Tomcat thus I cannot get access to its
> logs
>
> In general the connection lost is detected by what we call the PingThread (
>
> https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
> ) but not only
>
> https://ci-maven.apache.org/log/all
>
> For example it was few minutes ago we got 3 agents disconnected while
> nothing was running
>
> 2021-07-22 06:58:21.769+0000 [id=106291] INFO
> hudson.slaves.ChannelPinger$1#onDead:
> Ping failed. Terminating the channel maven4.
> java.util.concurrent.TimeoutException: Ping started at 1626936861769 hasn't
> completed by 1626937101769
> at hudson.remoting.PingThread.ping(PingThread.java:134)
> at hudson.remoting.PingThread.run(PingThread.java:90)
> 2021-07-22 06:58:21.778+0000 [id=106292] INFO
> hudson.slaves.ChannelPinger$1#onDead:
> Ping failed. Terminating the channel maven3.
> java.util.concurrent.TimeoutException: Ping started at 1626936861777 hasn't
> completed by 1626937101778
> at hudson.remoting.PingThread.ping(PingThread.java:134)
> at hudson.remoting.PingThread.run(PingThread.java:90)
> 2021-07-22 06:58:21.983+0000 [id=106295] INFO
> hudson.slaves.ChannelPinger$1#onDead:
> Ping failed. Terminating the channel maven5.
> java.util.concurrent.TimeoutException: Ping started at 1626936861982 hasn't
> completed by 1626937101983
> at hudson.remoting.PingThread.ping(PingThread.java:134)
> at hudson.remoting.PingThread.run(PingThread.java:90)
>
> @Gavin McDonald <gm...@apache.org> In terms of network, is it the same
> environment we use today compared to the ci-builds.apache.org environment
> ?
>
>
> On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <ti...@apache.org>
> wrote:
>
> > In my company, I also used 1GB for Xmx of Java Heap for the Jenkins JVM
> and
> > it was enough.
> > The subprocesses like Maven need to have much more memory to allocate for
> > themself rather than Jenkins JVM.
> > T
> >
> > On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com>
> > wrote:
> >
> > > I am looking at our builds and I try to understand why our agents are
> > often
> > > disconnected during the builds.
> > > We have in general a stacktrace like
> > >
> > > maven6 was marked offline: Connection was broken: java.io.IOException:
> > > Pipe closed after 0 cycles
> > > at
> > >
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> > > at
> > >
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> > > at
> > >
> >
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> > > at
> > >
> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> > > at
> > >
> >
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> > > at
> > >
> >
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> > > at
> > >
> >
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> > > at
> > >
> >
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
> > >
> > >
> > >
> > > As far I can see we are using 16Gb "hosts" for linux agents
> > >
> > > Something very strange is that the jenkins agent (this small component
> > > doing the link between the build host and the controller) is configured
> > > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
> > > (even more because of the non-heap)
> > > This one in general should require in general really less. 1Gb is
> > already a
> > > lot from my exp.
> > > Due to this, the OS can see it has the biggest process on the host and
> > > decide to kill it when the rest of the memory is used by the build.
> > > I think we should decrease this value.
> > > (I can do it but I don't know how was configured the ci.apache.org
> > agents
> > > and I would like to not add more issue if this setting was here in the
> > past
> > >
> > > I don't think it is the root cause of our instabilities (at least all)
> > and
> > > there is something else I have to find but it's a cheap fix to try
> > >
> > > FYI our agents VMs are ~like this today:
> > >
> > > - Java
> > > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> > > + Vendor: Oracle Corporation
> > > + Version: 1.8.0_291
> > > + Maximum memory: 7.67 GB (8232370176)
> > > + Allocated memory: 7.67 GB (8232370176)
> > > + Free memory: 6.03 GB (6470953760)
> > > + In-use memory: 1.64 GB (1761416416)
> > > + GC strategy: ParallelGC
> > > + Available CPUs: 4
> > >
> > > 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free
> mem
> > > is high)
> > >
> > > I would be in favor to try to launch them with -Xms128m
> > > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
> > >
> > > I think it's enough customization to start with
> > >
> > > Cheers
> > >
> > > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com>
> > > wrote:
> > >
> > > > I am not sure about the setup
> > > > AFAICS we don't use any JDK installer (
> > > > https://ci-maven.apache.org/configureTools/ ) thus I suppose that
> the
> > > > different JDKs are supposed to be installed directly on the agent ?
> > > > I am not sure how it was done on the previous environment
> > > >
> > > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <tibordigana@apache.org
> >
> > > > wrote:
> > > >
> > > >> The new CI system has the following issue:
> > > >>
> > > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> > > >>
> > > >>
> > > >>
> > >
> >
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <
> gmcdonald@apache.org>
> > > >> wrote:
> > > >>
> > > >> > Hi Maven folks.
> > > >> >
> > > >> > Infra has decided to separate off the Maven build jobs from
> > > >> > ci-builds.apache.org over to its very own Jenkins Controller and
> > > >> Agents.
> > > >> >
> > > >> > This means that Maven now has a dedicated Jenkins environment for
> > > >> itself.
> > > >> > It
> > > >> > also means that no other projects build jobs can build on the
> Maven
> > > >> nodes;
> > > >> > and
> > > >> > then Maven jobs will no longer be able to build on the ci-builds
> > > jobs.
> > > >> >
> > > >> > Your new Controller is set up as https://ci-maven.apache.org and
> > all
> > > >> Maven
> > > >> > Committers
> > > >> > can login via LDAP and create jobs.
> > > >> >
> > > >> > At the time of writing, there is one node/agent attached but I am
> > > >> building
> > > >> > 4 more - all
> > > >> > Ubuntu 20.04 and based in our Azure account.
> > > >> >
> > > >> > We can automagically move all your jobs over from ci-builds to
> > > ci-maven
> > > >> - I
> > > >> > just need someone to tell me go ahead and do it.
> > > >> >
> > > >> > In the meantime, feel free to have a test. The remaining 4 agents
> > will
> > > >> be
> > > >> > online
> > > >> > by tomorrow. We will review after a month if 5 is enough nodes.
> > > >> >
> > > >> > As with other projects having their own dedicated controller, who
> > have
> > > >> > taken advantage
> > > >> > of this isolation by having some nodes/agents given to the project
> > as
> > > a
> > > >> > 'targeted donation'
> > > >> > so someone here may know of a Company will to donate 5 - 10 or
> more
> > > >> nodes
> > > >> > specifically
> > > >> > for Maven Jenkins environment. Infra can afford to hand you over 5
> > > right
> > > >> > now.
> > > >> >
> > > >> > Let me know if you have any questions, otherwise let me know when
> I
> > > can
> > > >> > make the
> > > >> > transfer of your jobs.
> > > >> >
> > > >> > Thanks
> > > >> >
> > > >> > --
> > > >> >
> > > >> > *Gavin McDonald*
> > > >> > Systems Administrator
> > > >> > ASF Infrastructure Team
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > > Arnaud Héritier
> > > > Twitter/Skype : aheritier
> > > >
> > >
> > >
> > > --
> > > Arnaud Héritier
> > > Twitter/Skype : aheritier
> > >
> >
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
yes for the controller it depends of its size (number of jobs and types of
jobs) but here we are fine it seems with our 3Gb
* Java
- Version: 1.8.0_292
- Maximum memory: 3.00 GB (3221225472)
- Allocated memory: 3.00 GB (3221225472)
- Free memory: 750.15 MB (786591664)
- In-use memory: 2.27 GB (2434633808)
- GC strategy: G1
- Available CPUs: 2
For agents I reduced the memory allocated to the agent process but it
doesn't help much (it seems - even if it is still a good thing to do)
What is strange is that I see our agents sometimes disconnected even when
we have no activity on the jenkins controller
Sadly jenkins is deployed on Apache Tomcat thus I cannot get access to its
logs
In general the connection lost is detected by what we call the PingThread (
https://www.jenkins.io/doc/book/system-administration/monitoring/#ping-thread
) but not only
https://ci-maven.apache.org/log/all
For example it was few minutes ago we got 3 agents disconnected while
nothing was running
2021-07-22 06:58:21.769+0000 [id=106291] INFO
hudson.slaves.ChannelPinger$1#onDead:
Ping failed. Terminating the channel maven4.
java.util.concurrent.TimeoutException: Ping started at 1626936861769 hasn't
completed by 1626937101769
at hudson.remoting.PingThread.ping(PingThread.java:134)
at hudson.remoting.PingThread.run(PingThread.java:90)
2021-07-22 06:58:21.778+0000 [id=106292] INFO
hudson.slaves.ChannelPinger$1#onDead:
Ping failed. Terminating the channel maven3.
java.util.concurrent.TimeoutException: Ping started at 1626936861777 hasn't
completed by 1626937101778
at hudson.remoting.PingThread.ping(PingThread.java:134)
at hudson.remoting.PingThread.run(PingThread.java:90)
2021-07-22 06:58:21.983+0000 [id=106295] INFO
hudson.slaves.ChannelPinger$1#onDead:
Ping failed. Terminating the channel maven5.
java.util.concurrent.TimeoutException: Ping started at 1626936861982 hasn't
completed by 1626937101983
at hudson.remoting.PingThread.ping(PingThread.java:134)
at hudson.remoting.PingThread.run(PingThread.java:90)
@Gavin McDonald <gm...@apache.org> In terms of network, is it the same
environment we use today compared to the ci-builds.apache.org environment ?
On Wed, Jul 21, 2021 at 11:48 PM Tibor Digana <ti...@apache.org>
wrote:
> In my company, I also used 1GB for Xmx of Java Heap for the Jenkins JVM and
> it was enough.
> The subprocesses like Maven need to have much more memory to allocate for
> themself rather than Jenkins JVM.
> T
>
> On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
> > I am looking at our builds and I try to understand why our agents are
> often
> > disconnected during the builds.
> > We have in general a stacktrace like
> >
> > maven6 was marked offline: Connection was broken: java.io.IOException:
> > Pipe closed after 0 cycles
> > at
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> > at
> >
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> > at
> >
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> > at
> > hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> > at
> >
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> > at
> >
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> > at
> >
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> > at
> >
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
> >
> >
> >
> > As far I can see we are using 16Gb "hosts" for linux agents
> >
> > Something very strange is that the jenkins agent (this small component
> > doing the link between the build host and the controller) is configured
> > with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
> > (even more because of the non-heap)
> > This one in general should require in general really less. 1Gb is
> already a
> > lot from my exp.
> > Due to this, the OS can see it has the biggest process on the host and
> > decide to kill it when the rest of the memory is used by the build.
> > I think we should decrease this value.
> > (I can do it but I don't know how was configured the ci.apache.org
> agents
> > and I would like to not add more issue if this setting was here in the
> past
> >
> > I don't think it is the root cause of our instabilities (at least all)
> and
> > there is something else I have to find but it's a cheap fix to try
> >
> > FYI our agents VMs are ~like this today:
> >
> > - Java
> > + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> > + Vendor: Oracle Corporation
> > + Version: 1.8.0_291
> > + Maximum memory: 7.67 GB (8232370176)
> > + Allocated memory: 7.67 GB (8232370176)
> > + Free memory: 6.03 GB (6470953760)
> > + In-use memory: 1.64 GB (1761416416)
> > + GC strategy: ParallelGC
> > + Available CPUs: 4
> >
> > 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free mem
> > is high)
> >
> > I would be in favor to try to launch them with -Xms128m
> > -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
> >
> > I think it's enough customization to start with
> >
> > Cheers
> >
> > On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com>
> > wrote:
> >
> > > I am not sure about the setup
> > > AFAICS we don't use any JDK installer (
> > > https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
> > > different JDKs are supposed to be installed directly on the agent ?
> > > I am not sure how it was done on the previous environment
> > >
> > > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org>
> > > wrote:
> > >
> > >> The new CI system has the following issue:
> > >>
> > >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> > >>
> > >>
> > >>
> >
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> > >>
> > >>
> > >>
> > >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
> > >> wrote:
> > >>
> > >> > Hi Maven folks.
> > >> >
> > >> > Infra has decided to separate off the Maven build jobs from
> > >> > ci-builds.apache.org over to its very own Jenkins Controller and
> > >> Agents.
> > >> >
> > >> > This means that Maven now has a dedicated Jenkins environment for
> > >> itself.
> > >> > It
> > >> > also means that no other projects build jobs can build on the Maven
> > >> nodes;
> > >> > and
> > >> > then Maven jobs will no longer be able to build on the ci-builds
> > jobs.
> > >> >
> > >> > Your new Controller is set up as https://ci-maven.apache.org and
> all
> > >> Maven
> > >> > Committers
> > >> > can login via LDAP and create jobs.
> > >> >
> > >> > At the time of writing, there is one node/agent attached but I am
> > >> building
> > >> > 4 more - all
> > >> > Ubuntu 20.04 and based in our Azure account.
> > >> >
> > >> > We can automagically move all your jobs over from ci-builds to
> > ci-maven
> > >> - I
> > >> > just need someone to tell me go ahead and do it.
> > >> >
> > >> > In the meantime, feel free to have a test. The remaining 4 agents
> will
> > >> be
> > >> > online
> > >> > by tomorrow. We will review after a month if 5 is enough nodes.
> > >> >
> > >> > As with other projects having their own dedicated controller, who
> have
> > >> > taken advantage
> > >> > of this isolation by having some nodes/agents given to the project
> as
> > a
> > >> > 'targeted donation'
> > >> > so someone here may know of a Company will to donate 5 - 10 or more
> > >> nodes
> > >> > specifically
> > >> > for Maven Jenkins environment. Infra can afford to hand you over 5
> > right
> > >> > now.
> > >> >
> > >> > Let me know if you have any questions, otherwise let me know when I
> > can
> > >> > make the
> > >> > transfer of your jobs.
> > >> >
> > >> > Thanks
> > >> >
> > >> > --
> > >> >
> > >> > *Gavin McDonald*
> > >> > Systems Administrator
> > >> > ASF Infrastructure Team
> > >> >
> > >>
> > >
> > >
> > > --
> > > Arnaud Héritier
> > > Twitter/Skype : aheritier
> > >
> >
> >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Tibor Digana <ti...@apache.org>.
In my company, I also used 1GB for Xmx of Java Heap for the Jenkins JVM and
it was enough.
The subprocesses like Maven need to have much more memory to allocate for
themself rather than Jenkins JVM.
T
On Wed, Jul 21, 2021 at 6:38 PM Arnaud Héritier <ah...@gmail.com> wrote:
> I am looking at our builds and I try to understand why our agents are often
> disconnected during the builds.
> We have in general a stacktrace like
>
> maven6 was marked offline: Connection was broken: java.io.IOException:
> Pipe closed after 0 cycles
> at
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
> at
> org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
> at
> hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
> at
> hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
> at
> hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
> at
> hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
> at
> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
> at
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
>
>
>
> As far I can see we are using 16Gb "hosts" for linux agents
>
> Something very strange is that the jenkins agent (this small component
> doing the link between the build host and the controller) is configured
> with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
> (even more because of the non-heap)
> This one in general should require in general really less. 1Gb is already a
> lot from my exp.
> Due to this, the OS can see it has the biggest process on the host and
> decide to kill it when the rest of the memory is used by the build.
> I think we should decrease this value.
> (I can do it but I don't know how was configured the ci.apache.org agents
> and I would like to not add more issue if this setting was here in the past
>
> I don't think it is the root cause of our instabilities (at least all) and
> there is something else I have to find but it's a cheap fix to try
>
> FYI our agents VMs are ~like this today:
>
> - Java
> + Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
> + Vendor: Oracle Corporation
> + Version: 1.8.0_291
> + Maximum memory: 7.67 GB (8232370176)
> + Allocated memory: 7.67 GB (8232370176)
> + Free memory: 6.03 GB (6470953760)
> + In-use memory: 1.64 GB (1761416416)
> + GC strategy: ParallelGC
> + Available CPUs: 4
>
> 8Gb is reserved, 1Gb is used (because the GC does nothing as the Free mem
> is high)
>
> I would be in favor to try to launch them with -Xms128m
> -Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
>
> I think it's enough customization to start with
>
> Cheers
>
> On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com>
> wrote:
>
> > I am not sure about the setup
> > AFAICS we don't use any JDK installer (
> > https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
> > different JDKs are supposed to be installed directly on the agent ?
> > I am not sure how it was done on the previous environment
> >
> > On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org>
> > wrote:
> >
> >> The new CI system has the following issue:
> >>
> >> /home/jenkins/tools/java/latest1.7/bin/java: not found
> >>
> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
> >>
> >>
> >>
> >> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
> >> wrote:
> >>
> >> > Hi Maven folks.
> >> >
> >> > Infra has decided to separate off the Maven build jobs from
> >> > ci-builds.apache.org over to its very own Jenkins Controller and
> >> Agents.
> >> >
> >> > This means that Maven now has a dedicated Jenkins environment for
> >> itself.
> >> > It
> >> > also means that no other projects build jobs can build on the Maven
> >> nodes;
> >> > and
> >> > then Maven jobs will no longer be able to build on the ci-builds
> jobs.
> >> >
> >> > Your new Controller is set up as https://ci-maven.apache.org and all
> >> Maven
> >> > Committers
> >> > can login via LDAP and create jobs.
> >> >
> >> > At the time of writing, there is one node/agent attached but I am
> >> building
> >> > 4 more - all
> >> > Ubuntu 20.04 and based in our Azure account.
> >> >
> >> > We can automagically move all your jobs over from ci-builds to
> ci-maven
> >> - I
> >> > just need someone to tell me go ahead and do it.
> >> >
> >> > In the meantime, feel free to have a test. The remaining 4 agents will
> >> be
> >> > online
> >> > by tomorrow. We will review after a month if 5 is enough nodes.
> >> >
> >> > As with other projects having their own dedicated controller, who have
> >> > taken advantage
> >> > of this isolation by having some nodes/agents given to the project as
> a
> >> > 'targeted donation'
> >> > so someone here may know of a Company will to donate 5 - 10 or more
> >> nodes
> >> > specifically
> >> > for Maven Jenkins environment. Infra can afford to hand you over 5
> right
> >> > now.
> >> >
> >> > Let me know if you have any questions, otherwise let me know when I
> can
> >> > make the
> >> > transfer of your jobs.
> >> >
> >> > Thanks
> >> >
> >> > --
> >> >
> >> > *Gavin McDonald*
> >> > Systems Administrator
> >> > ASF Infrastructure Team
> >> >
> >>
> >
> >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
I am looking at our builds and I try to understand why our agents are often
disconnected during the builds.
We have in general a stacktrace like
maven6 was marked offline: Connection was broken: java.io.IOException:
Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
As far I can see we are using 16Gb "hosts" for linux agents
Something very strange is that the jenkins agent (this small component
doing the link between the build host and the controller) is configured
with `-Xms8g -Xmx8g` thus we are reserving for it 50% of the server mem
(even more because of the non-heap)
This one in general should require in general really less. 1Gb is already a
lot from my exp.
Due to this, the OS can see it has the biggest process on the host and
decide to kill it when the rest of the memory is used by the build.
I think we should decrease this value.
(I can do it but I don't know how was configured the ci.apache.org agents
and I would like to not add more issue if this setting was here in the past
I don't think it is the root cause of our instabilities (at least all) and
there is something else I have to find but it's a cheap fix to try
FYI our agents VMs are ~like this today:
- Java
+ Home: `/usr/local/asfpackages/java/oraclejdk-1.8.0-291/jre`
+ Vendor: Oracle Corporation
+ Version: 1.8.0_291
+ Maximum memory: 7.67 GB (8232370176)
+ Allocated memory: 7.67 GB (8232370176)
+ Free memory: 6.03 GB (6470953760)
+ In-use memory: 1.64 GB (1761416416)
+ GC strategy: ParallelGC
+ Available CPUs: 4
8Gb is reserved, 1Gb is used (because the GC does nothing as the Free mem
is high)
I would be in favor to try to launch them with -Xms128m
-Xmx1g -XX:+UseG1GC -XX:+UseStringDeduplication
I think it's enough customization to start with
Cheers
On Wed, Jul 21, 2021 at 1:28 PM Arnaud Héritier <ah...@gmail.com> wrote:
> I am not sure about the setup
> AFAICS we don't use any JDK installer (
> https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
> different JDKs are supposed to be installed directly on the agent ?
> I am not sure how it was done on the previous environment
>
> On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org>
> wrote:
>
>> The new CI system has the following issue:
>>
>> /home/jenkins/tools/java/latest1.7/bin/java: not found
>>
>>
>> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>>
>>
>>
>> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
>> wrote:
>>
>> > Hi Maven folks.
>> >
>> > Infra has decided to separate off the Maven build jobs from
>> > ci-builds.apache.org over to its very own Jenkins Controller and
>> Agents.
>> >
>> > This means that Maven now has a dedicated Jenkins environment for
>> itself.
>> > It
>> > also means that no other projects build jobs can build on the Maven
>> nodes;
>> > and
>> > then Maven jobs will no longer be able to build on the ci-builds jobs.
>> >
>> > Your new Controller is set up as https://ci-maven.apache.org and all
>> Maven
>> > Committers
>> > can login via LDAP and create jobs.
>> >
>> > At the time of writing, there is one node/agent attached but I am
>> building
>> > 4 more - all
>> > Ubuntu 20.04 and based in our Azure account.
>> >
>> > We can automagically move all your jobs over from ci-builds to ci-maven
>> - I
>> > just need someone to tell me go ahead and do it.
>> >
>> > In the meantime, feel free to have a test. The remaining 4 agents will
>> be
>> > online
>> > by tomorrow. We will review after a month if 5 is enough nodes.
>> >
>> > As with other projects having their own dedicated controller, who have
>> > taken advantage
>> > of this isolation by having some nodes/agents given to the project as a
>> > 'targeted donation'
>> > so someone here may know of a Company will to donate 5 - 10 or more
>> nodes
>> > specifically
>> > for Maven Jenkins environment. Infra can afford to hand you over 5 right
>> > now.
>> >
>> > Let me know if you have any questions, otherwise let me know when I can
>> > make the
>> > transfer of your jobs.
>> >
>> > Thanks
>> >
>> > --
>> >
>> > *Gavin McDonald*
>> > Systems Administrator
>> > ASF Infrastructure Team
>> >
>>
>
>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
I am not sure about the setup
AFAICS we don't use any JDK installer (
https://ci-maven.apache.org/configureTools/ ) thus I suppose that the
different JDKs are supposed to be installed directly on the agent ?
I am not sure how it was done on the previous environment
On Sun, Jul 18, 2021 at 5:30 PM Tibor Digana <ti...@apache.org> wrote:
> The new CI system has the following issue:
>
> /home/jenkins/tools/java/latest1.7/bin/java: not found
>
>
> https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
>
>
>
> On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org>
> wrote:
>
> > Hi Maven folks.
> >
> > Infra has decided to separate off the Maven build jobs from
> > ci-builds.apache.org over to its very own Jenkins Controller and Agents.
> >
> > This means that Maven now has a dedicated Jenkins environment for itself.
> > It
> > also means that no other projects build jobs can build on the Maven
> nodes;
> > and
> > then Maven jobs will no longer be able to build on the ci-builds jobs.
> >
> > Your new Controller is set up as https://ci-maven.apache.org and all
> Maven
> > Committers
> > can login via LDAP and create jobs.
> >
> > At the time of writing, there is one node/agent attached but I am
> building
> > 4 more - all
> > Ubuntu 20.04 and based in our Azure account.
> >
> > We can automagically move all your jobs over from ci-builds to ci-maven
> - I
> > just need someone to tell me go ahead and do it.
> >
> > In the meantime, feel free to have a test. The remaining 4 agents will be
> > online
> > by tomorrow. We will review after a month if 5 is enough nodes.
> >
> > As with other projects having their own dedicated controller, who have
> > taken advantage
> > of this isolation by having some nodes/agents given to the project as a
> > 'targeted donation'
> > so someone here may know of a Company will to donate 5 - 10 or more nodes
> > specifically
> > for Maven Jenkins environment. Infra can afford to hand you over 5 right
> > now.
> >
> > Let me know if you have any questions, otherwise let me know when I can
> > make the
> > transfer of your jobs.
> >
> > Thanks
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
> >
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Tibor Digana <ti...@apache.org>.
The new CI system has the following issue:
/home/jenkins/tools/java/latest1.7/bin/java: not found
https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/104/execution/node/183/log/
On Wed, Jun 30, 2021 at 8:03 PM Gavin McDonald <gm...@apache.org> wrote:
> Hi Maven folks.
>
> Infra has decided to separate off the Maven build jobs from
> ci-builds.apache.org over to its very own Jenkins Controller and Agents.
>
> This means that Maven now has a dedicated Jenkins environment for itself.
> It
> also means that no other projects build jobs can build on the Maven nodes;
> and
> then Maven jobs will no longer be able to build on the ci-builds jobs.
>
> Your new Controller is set up as https://ci-maven.apache.org and all Maven
> Committers
> can login via LDAP and create jobs.
>
> At the time of writing, there is one node/agent attached but I am building
> 4 more - all
> Ubuntu 20.04 and based in our Azure account.
>
> We can automagically move all your jobs over from ci-builds to ci-maven - I
> just need someone to tell me go ahead and do it.
>
> In the meantime, feel free to have a test. The remaining 4 agents will be
> online
> by tomorrow. We will review after a month if 5 is enough nodes.
>
> As with other projects having their own dedicated controller, who have
> taken advantage
> of this isolation by having some nodes/agents given to the project as a
> 'targeted donation'
> so someone here may know of a Company will to donate 5 - 10 or more nodes
> specifically
> for Maven Jenkins environment. Infra can afford to hand you over 5 right
> now.
>
> Let me know if you have any questions, otherwise let me know when I can
> make the
> transfer of your jobs.
>
> Thanks
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi
On Thu, Jul 15, 2021 at 10:29 AM Robert Scholte <rf...@apache.org>
wrote:
> Our scripts use the labels ubuntu and Windows to distinguish the OS.
> Is it safe to add ubuntu to the maven nodes?
>
> Sure, and done.
thanks,
> Robert
> On 15-7-2021 10:06:37, Gavin McDonald <gm...@apache.org> wrote:
> Thanks All,
>
> Jobs have been migrated.
>
> Please test and report any issues ASAP to INFRA jira.
>
> https://ci-maven.apache.org/
>
>
>
> On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > Hi Gavin
> >
> > Let’s go. I cannot test more than the authentication today but I can
> assist
> > more later this month after my vacations.
> >
> > What is important is to move the shared libraries used by our jobs (I do
> > not remember how they were configured).
> >
> > We will see if we need / have to find some sponsors to setup some
> > additional build nodes.
> >
> > Thanks for your help.
> >
> > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > écrit :
> >
> > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > gmcdonald@apache.org>
> > > ha scritto:
> > >
> > > > Hi All,
> > > >
> > > > With no more interest in this subject, and no willing testers, I'll
> be
> > > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > > >
> > >
> > > I have tested that I am able to login to ci-maven with my LDAP
> credentials
> > >
> > > Thanks
> > > Enrico
> > >
> > >
> > > >
> > > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > > anything that breaks as a result.
> > > >
> > > > Gav...
> > > >
> > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > Hi Michael,
> > > > >
> > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > Hi Maven folks.
> > > > > > >
> > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > ci-builds.apache.org over to its very own Jenkins Controller
> and
> > > > Agents.
> > > > > > >
> > > > > > > This means that Maven now has a dedicated Jenkins environment
> for
> > > > itself.
> > > > > > > It
> > > > > > > also means that no other projects build jobs can build on the
> Maven
> > > > nodes;
> > > > > > > and
> > > > > > > then Maven jobs will no longer be able to build on the
> ci-builds
> > > > jobs.
> > > > > > >
> > > > > > > Your new Controller is set up as https://ci-maven.apache.org
> and
> > > > all Maven
> > > > > > > Committers
> > > > > > > can login via LDAP and create jobs.
> > > > > > >
> > > > > > > At the time of writing, there is one node/agent attached but I
> am
> > > > building
> > > > > > > 4 more - all
> > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > >
> > > > > > Well, just Ubuntu? We actually need at least three different
> > > operating
> > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > Since this is on an Azure can you rather add at least one
> Windows and
> > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > >
> > > > > You are getting exactly as you have been using on ci-builds.a.o
> (and
> > > > builds.a.o) before that.
> > > > > There is no FreeBSD on any of our CI nor has there been for many
> years,
> > > > we do not support
> > > > > it.
> > > > >
> > > > > As for Windows nodes, you still have access to the same Windows
> nodes
> > > as
> > > > you always have, they are what are called floating agents available
> for
> > > > lease by all Jenkins Controllers, so your access and availability to
> the
> > > > Windows nodes will not change. We have 6 Windoes nodes currently and
> plan
> > > > to add some more soon.
> > > > >
> > > > > If anyone were to donate any nodes to attach to ci-maven then we'd
> be
> > > > happy to help add them on, including FreeBSD donated nodes, only
> > > difference
> > > > being that Infra would not manage them nor install any software on
> them,
> > > > which would be up to the donator(s).
> > > > >
> > > > > HTH
> > > > >
> > > > > >
> > > > > > Michael
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
This was indeed the issue.
Gavin has configured the plugin and now builds start to turn blue/green again.
I'll try to get trigger all master and get these back to blue[1]
If you see any issues, please reply to this mail.
Robert
[1] https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
On 16-7-2021 13:38:53, Robert Scholte <rf...@apache.org> wrote:
Could this be the issue?
[Jira] No Jira site is configured for this project. This must be a project configuration error
@Gavin, could you configure this like it was done with ci-builds? Especially the credentials seem to be an issue for me.
thanks,
Robert
On 16-7-2021 12:49:30, Robert Scholte <rf...@apache.org> wrote:
Gavin,
I've been able to install the required plugin.
I think we should be able to fix the rest ourselves.
@Arnaud and others
https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-acr-plugin/job/master/
All stages succeed, but in the end the build is marked as failure.
I can't see the cause, so I could use extra help.
thanks,
Robert
On 16-7-2021 11:22:12, Gavin McDonald <gm...@apache.org> wrote:
All PMC Members should see an 'Administer' button down the leftside nav
On Fri, Jul 16, 2021 at 10:38 AM Robert Scholte
wrote:
> This would indeed be a good one for Arnaud.
> I don't recognize an option related to this.
> What do you expect me to see, or can you share a link to see if I have
> access?
>
> thanks,
> Robert
>
> On 16-7-2021 10:25:04, Gavin McDonald wrote:
> Hi
>
> I think I have it configured correctly that PMC can manage the controller,
> can someone test by
> trying to install this plugin and let me know?
>
> TIA
>
> On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier
> wrote:
>
> > This one:
> > https://plugins.jenkins.io/ws-cleanup/
> >
> > Do we (at least PMCs) have admin privileges on the controller? In such
> > case you can just add it from the plugins management admin panel.
> >
> > You just search the missing keyword on jenkins.io to find the plugin
> > implementing the missing command.
> >
> > We should probably document these needs somewhere.
> >
> > Arnaud
> >
> > Le ven. 16 juil. 2021 à 10:02, Robert Scholte a
> > écrit :
> >
> >> Yes, that fixed the build generating pages like
> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
> >>
> >>
> >> Most masters are broken, most likely all due to this error:"
> >> No such DSL method 'cleanWs'
> >>
> >>
> >> Looks like another plugin is missing.
> >>
> >> Robert
> >> On 16-7-2021 09:52:15, Gavin McDonald wrote:
> >> Thanks Robert,
> >>
> >> I believe those are a result of missing plugins Ansicolor and
> Timestamper.
> >>
> >> Both now installed
> >>
> >> HTH
> >>
> >> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
> >>
> >> > Okay, that makes sense.
> >> > Jobs have been picked up, just required quite some hours to run
> >> everything.
> >> >
> >> > this morning I noticed some failed with the following message:
> >> >
> >> > org.codehaus.groovy.control.MultipleCompilationErrorsException:
> startup
> >> failed:
> >> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> >> > timestamps()
> >> > ^
> >> >
> >> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> >> > ansiColor('xterm')
> >> > ^
> >> >
> >> > 2 errors
> >> >
> >> > at
> >>
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>
> >> > at
> >> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> >> > at
> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> >> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> >> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> >> > at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > at hudson.model.Executor.run(Executor.java:429)
> >> > Finished: FAILURE
> >> >
> >> >
> >> >
> >> > I've looked into our Jenkins scripts and and can't find these
> options.
> >> So
> >> > I assume these are some globally defined scripts that needs to be
> >> repaired.
> >> >
> >> > thanks,
> >> > Robert
> >> >
> >> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
> >> > Hi
> >> >
> >> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> >> > wrote:
> >> >
> >> > > In case you haven't noticed: all Windows nodes are offline.
> >> > >
> >> > >
> >> > Yeah they are 'leased' nodes shared between all ci-* controllers and
> >> come
> >> > online
> >> > when in use.
> >> >
> >> >
> >> > Robert
> >> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
> >> > > Our scripts use the labels ubuntu and Windows to distinguish the
> OS.
> >> > > Is it safe to add ubuntu to the maven nodes?
> >> > >
> >> > > thanks,
> >> > > Robert
> >> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> >> > > Thanks All,
> >> > >
> >> > > Jobs have been migrated.
> >> > >
> >> > > Please test and report any issues ASAP to INFRA jira.
> >> > >
> >> > > https://ci-maven.apache.org/
> >> > >
> >> > >
> >> > >
> >> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> >> > > > Hi Gavin
> >> > > >
> >> > > > Let’s go. I cannot test more than the authentication today but I
> can
> >> > > assist
> >> > > > more later this month after my vacations.
> >> > > >
> >> > > > What is important is to move the shared libraries used by our
> jobs
> >> (I
> >> > do
> >> > > > not remember how they were configured).
> >> > > >
> >> > > > We will see if we need / have to find some sponsors to setup some
> >> > > > additional build nodes.
> >> > > >
> >> > > > Thanks for your help.
> >> > > >
> >> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> >> > > > écrit :
> >> > > >
> >> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> >> > > > > gmcdonald@apache.org>
> >> > > > > ha scritto:
> >> > > > >
> >> > > > > > Hi All,
> >> > > > > >
> >> > > > > > With no more interest in this subject, and no willing
> testers,
> >> > I'll
> >> > > be
> >> > > > > > going ahead tomorrow and move all Maven jobs to
> >> > ci-maven.apache.org.
> >> > > > > >
> >> > > > >
> >> > > > > I have tested that I am able to login to ci-maven with my LDAP
> >> > > credentials
> >> > > > >
> >> > > > > Thanks
> >> > > > > Enrico
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > Feel free to check after the move and create INFRA jira
> >> ticket(s)
> >> > for
> >> > > > > > anything that breaks as a result.
> >> > > > > >
> >> > > > > > Gav...
> >> > > > > >
> >> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> >> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> >> > > > > > > > > Hi Maven folks.
> >> > > > > > > > >
> >> > > > > > > > > Infra has decided to separate off the Maven build jobs
> >> from
> >> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
> >> > Controller
> >> > > and
> >> > > > > > Agents.
> >> > > > > > > > >
> >> > > > > > > > > This means that Maven now has a dedicated Jenkins
> >> > environment
> >> > > for
> >> > > > > > itself.
> >> > > > > > > > > It
> >> > > > > > > > > also means that no other projects build jobs can build
> on
> >> > the
> >> > > Maven
> >> > > > > > nodes;
> >> > > > > > > > > and
> >> > > > > > > > > then Maven jobs will no longer be able to build on the
> >> > > ci-builds
> >> > > > > > jobs.
> >> > > > > > > > >
> >> > > > > > > > > Your new Controller is set up as
> >> https://ci-maven.apache.org
> >> > > and
> >> > > > > > all Maven
> >> > > > > > > > > Committers
> >> > > > > > > > > can login via LDAP and create jobs.
> >> > > > > > > > >
> >> > > > > > > > > At the time of writing, there is one node/agent
> attached
> >> but
> >> > I
> >> > > am
> >> > > > > > building
> >> > > > > > > > > 4 more - all
> >> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
> >> > > > > > > >
> >> > > > > > > > Well, just Ubuntu? We actually need at least three
> different
> >> > > > > operating
> >> > > > > > > > systems to detect subtile bugs which I have found over
> time.
> >> > > > > > > > Since this is on an Azure can you rather add at least one
> >> > > Windows and
> >> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> >> > > > > > >
> >> > > > > > > You are getting exactly as you have been using on
> >> ci-builds.a.o
> >> > > (and
> >> > > > > > builds.a.o) before that.
> >> > > > > > > There is no FreeBSD on any of our CI nor has there been for
> >> many
> >> > > years,
> >> > > > > > we do not support
> >> > > > > > > it.
> >> > > > > > >
> >> > > > > > > As for Windows nodes, you still have access to the same
> >> Windows
> >> > > nodes
> >> > > > > as
> >> > > > > > you always have, they are what are called floating agents
> >> > available
> >> > > for
> >> > > > > > lease by all Jenkins Controllers, so your access and
> >> availability
> >> > to
> >> > > the
> >> > > > > > Windows nodes will not change. We have 6 Windoes nodes
> currently
> >> > and
> >> > > plan
> >> > > > > > to add some more soon.
> >> > > > > > >
> >> > > > > > > If anyone were to donate any nodes to attach to ci-maven
> then
> >> > we'd
> >> > > be
> >> > > > > > happy to help add them on, including FreeBSD donated nodes,
> only
> >> > > > > difference
> >> > > > > > being that Infra would not manage them nor install any
> software
> >> on
> >> > > them,
> >> > > > > > which would be up to the donator(s).
> >> > > > > > >
> >> > > > > > > HTH
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > > For additional commands, e-mail:
> dev-help@maven.apache.org
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > ---------------------------------------------------------------------
> >> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > --
> >> > > > Arnaud Héritier
> >> > > > Twitter/Skype : aheritier
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > >
> >> > >
> >> >
> >> > --
> >> >
> >> > *Gavin McDonald*
> >> > Systems Administrator
> >> > ASF Infrastructure Team
> >> >
> >> >
> >>
> >> --
> >>
> >> *Gavin McDonald*
> >> Systems Administrator
> >> ASF Infrastructure Team
> >>
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
Could this be the issue?
[Jira] No Jira site is configured for this project. This must be a project configuration error
@Gavin, could you configure this like it was done with ci-builds? Especially the credentials seem to be an issue for me.
thanks,
Robert
On 16-7-2021 12:49:30, Robert Scholte <rf...@apache.org> wrote:
Gavin,
I've been able to install the required plugin.
I think we should be able to fix the rest ourselves.
@Arnaud and others
https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-acr-plugin/job/master/
All stages succeed, but in the end the build is marked as failure.
I can't see the cause, so I could use extra help.
thanks,
Robert
On 16-7-2021 11:22:12, Gavin McDonald <gm...@apache.org> wrote:
All PMC Members should see an 'Administer' button down the leftside nav
On Fri, Jul 16, 2021 at 10:38 AM Robert Scholte
wrote:
> This would indeed be a good one for Arnaud.
> I don't recognize an option related to this.
> What do you expect me to see, or can you share a link to see if I have
> access?
>
> thanks,
> Robert
>
> On 16-7-2021 10:25:04, Gavin McDonald wrote:
> Hi
>
> I think I have it configured correctly that PMC can manage the controller,
> can someone test by
> trying to install this plugin and let me know?
>
> TIA
>
> On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier
> wrote:
>
> > This one:
> > https://plugins.jenkins.io/ws-cleanup/
> >
> > Do we (at least PMCs) have admin privileges on the controller? In such
> > case you can just add it from the plugins management admin panel.
> >
> > You just search the missing keyword on jenkins.io to find the plugin
> > implementing the missing command.
> >
> > We should probably document these needs somewhere.
> >
> > Arnaud
> >
> > Le ven. 16 juil. 2021 à 10:02, Robert Scholte a
> > écrit :
> >
> >> Yes, that fixed the build generating pages like
> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
> >>
> >>
> >> Most masters are broken, most likely all due to this error:"
> >> No such DSL method 'cleanWs'
> >>
> >>
> >> Looks like another plugin is missing.
> >>
> >> Robert
> >> On 16-7-2021 09:52:15, Gavin McDonald wrote:
> >> Thanks Robert,
> >>
> >> I believe those are a result of missing plugins Ansicolor and
> Timestamper.
> >>
> >> Both now installed
> >>
> >> HTH
> >>
> >> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
> >>
> >> > Okay, that makes sense.
> >> > Jobs have been picked up, just required quite some hours to run
> >> everything.
> >> >
> >> > this morning I noticed some failed with the following message:
> >> >
> >> > org.codehaus.groovy.control.MultipleCompilationErrorsException:
> startup
> >> failed:
> >> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> >> > timestamps()
> >> > ^
> >> >
> >> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> >> > ansiColor('xterm')
> >> > ^
> >> >
> >> > 2 errors
> >> >
> >> > at
> >>
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>
> >> > at
> >> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> >> > at
> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> >> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> >> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> >> > at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > at hudson.model.Executor.run(Executor.java:429)
> >> > Finished: FAILURE
> >> >
> >> >
> >> >
> >> > I've looked into our Jenkins scripts and and can't find these
> options.
> >> So
> >> > I assume these are some globally defined scripts that needs to be
> >> repaired.
> >> >
> >> > thanks,
> >> > Robert
> >> >
> >> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
> >> > Hi
> >> >
> >> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> >> > wrote:
> >> >
> >> > > In case you haven't noticed: all Windows nodes are offline.
> >> > >
> >> > >
> >> > Yeah they are 'leased' nodes shared between all ci-* controllers and
> >> come
> >> > online
> >> > when in use.
> >> >
> >> >
> >> > Robert
> >> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
> >> > > Our scripts use the labels ubuntu and Windows to distinguish the
> OS.
> >> > > Is it safe to add ubuntu to the maven nodes?
> >> > >
> >> > > thanks,
> >> > > Robert
> >> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> >> > > Thanks All,
> >> > >
> >> > > Jobs have been migrated.
> >> > >
> >> > > Please test and report any issues ASAP to INFRA jira.
> >> > >
> >> > > https://ci-maven.apache.org/
> >> > >
> >> > >
> >> > >
> >> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> >> > > > Hi Gavin
> >> > > >
> >> > > > Let’s go. I cannot test more than the authentication today but I
> can
> >> > > assist
> >> > > > more later this month after my vacations.
> >> > > >
> >> > > > What is important is to move the shared libraries used by our
> jobs
> >> (I
> >> > do
> >> > > > not remember how they were configured).
> >> > > >
> >> > > > We will see if we need / have to find some sponsors to setup some
> >> > > > additional build nodes.
> >> > > >
> >> > > > Thanks for your help.
> >> > > >
> >> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> >> > > > écrit :
> >> > > >
> >> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> >> > > > > gmcdonald@apache.org>
> >> > > > > ha scritto:
> >> > > > >
> >> > > > > > Hi All,
> >> > > > > >
> >> > > > > > With no more interest in this subject, and no willing
> testers,
> >> > I'll
> >> > > be
> >> > > > > > going ahead tomorrow and move all Maven jobs to
> >> > ci-maven.apache.org.
> >> > > > > >
> >> > > > >
> >> > > > > I have tested that I am able to login to ci-maven with my LDAP
> >> > > credentials
> >> > > > >
> >> > > > > Thanks
> >> > > > > Enrico
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > Feel free to check after the move and create INFRA jira
> >> ticket(s)
> >> > for
> >> > > > > > anything that breaks as a result.
> >> > > > > >
> >> > > > > > Gav...
> >> > > > > >
> >> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> >> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> >> > > > > > > > > Hi Maven folks.
> >> > > > > > > > >
> >> > > > > > > > > Infra has decided to separate off the Maven build jobs
> >> from
> >> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
> >> > Controller
> >> > > and
> >> > > > > > Agents.
> >> > > > > > > > >
> >> > > > > > > > > This means that Maven now has a dedicated Jenkins
> >> > environment
> >> > > for
> >> > > > > > itself.
> >> > > > > > > > > It
> >> > > > > > > > > also means that no other projects build jobs can build
> on
> >> > the
> >> > > Maven
> >> > > > > > nodes;
> >> > > > > > > > > and
> >> > > > > > > > > then Maven jobs will no longer be able to build on the
> >> > > ci-builds
> >> > > > > > jobs.
> >> > > > > > > > >
> >> > > > > > > > > Your new Controller is set up as
> >> https://ci-maven.apache.org
> >> > > and
> >> > > > > > all Maven
> >> > > > > > > > > Committers
> >> > > > > > > > > can login via LDAP and create jobs.
> >> > > > > > > > >
> >> > > > > > > > > At the time of writing, there is one node/agent
> attached
> >> but
> >> > I
> >> > > am
> >> > > > > > building
> >> > > > > > > > > 4 more - all
> >> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
> >> > > > > > > >
> >> > > > > > > > Well, just Ubuntu? We actually need at least three
> different
> >> > > > > operating
> >> > > > > > > > systems to detect subtile bugs which I have found over
> time.
> >> > > > > > > > Since this is on an Azure can you rather add at least one
> >> > > Windows and
> >> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> >> > > > > > >
> >> > > > > > > You are getting exactly as you have been using on
> >> ci-builds.a.o
> >> > > (and
> >> > > > > > builds.a.o) before that.
> >> > > > > > > There is no FreeBSD on any of our CI nor has there been for
> >> many
> >> > > years,
> >> > > > > > we do not support
> >> > > > > > > it.
> >> > > > > > >
> >> > > > > > > As for Windows nodes, you still have access to the same
> >> Windows
> >> > > nodes
> >> > > > > as
> >> > > > > > you always have, they are what are called floating agents
> >> > available
> >> > > for
> >> > > > > > lease by all Jenkins Controllers, so your access and
> >> availability
> >> > to
> >> > > the
> >> > > > > > Windows nodes will not change. We have 6 Windoes nodes
> currently
> >> > and
> >> > > plan
> >> > > > > > to add some more soon.
> >> > > > > > >
> >> > > > > > > If anyone were to donate any nodes to attach to ci-maven
> then
> >> > we'd
> >> > > be
> >> > > > > > happy to help add them on, including FreeBSD donated nodes,
> only
> >> > > > > difference
> >> > > > > > being that Infra would not manage them nor install any
> software
> >> on
> >> > > them,
> >> > > > > > which would be up to the donator(s).
> >> > > > > > >
> >> > > > > > > HTH
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > > For additional commands, e-mail:
> dev-help@maven.apache.org
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > ---------------------------------------------------------------------
> >> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > --
> >> > > > Arnaud Héritier
> >> > > > Twitter/Skype : aheritier
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > >
> >> > >
> >> >
> >> > --
> >> >
> >> > *Gavin McDonald*
> >> > Systems Administrator
> >> > ASF Infrastructure Team
> >> >
> >> >
> >>
> >> --
> >>
> >> *Gavin McDonald*
> >> Systems Administrator
> >> ASF Infrastructure Team
> >>
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
Gavin,
I've been able to install the required plugin.
I think we should be able to fix the rest ourselves.
@Arnaud and others
https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-acr-plugin/job/master/
All stages succeed, but in the end the build is marked as failure.
I can't see the cause, so I could use extra help.
thanks,
Robert
On 16-7-2021 11:22:12, Gavin McDonald <gm...@apache.org> wrote:
All PMC Members should see an 'Administer' button down the leftside nav
On Fri, Jul 16, 2021 at 10:38 AM Robert Scholte
wrote:
> This would indeed be a good one for Arnaud.
> I don't recognize an option related to this.
> What do you expect me to see, or can you share a link to see if I have
> access?
>
> thanks,
> Robert
>
> On 16-7-2021 10:25:04, Gavin McDonald wrote:
> Hi
>
> I think I have it configured correctly that PMC can manage the controller,
> can someone test by
> trying to install this plugin and let me know?
>
> TIA
>
> On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier
> wrote:
>
> > This one:
> > https://plugins.jenkins.io/ws-cleanup/
> >
> > Do we (at least PMCs) have admin privileges on the controller? In such
> > case you can just add it from the plugins management admin panel.
> >
> > You just search the missing keyword on jenkins.io to find the plugin
> > implementing the missing command.
> >
> > We should probably document these needs somewhere.
> >
> > Arnaud
> >
> > Le ven. 16 juil. 2021 à 10:02, Robert Scholte a
> > écrit :
> >
> >> Yes, that fixed the build generating pages like
> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
> >>
> >>
> >> Most masters are broken, most likely all due to this error:"
> >> No such DSL method 'cleanWs'
> >>
> >>
> >> Looks like another plugin is missing.
> >>
> >> Robert
> >> On 16-7-2021 09:52:15, Gavin McDonald wrote:
> >> Thanks Robert,
> >>
> >> I believe those are a result of missing plugins Ansicolor and
> Timestamper.
> >>
> >> Both now installed
> >>
> >> HTH
> >>
> >> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
> >>
> >> > Okay, that makes sense.
> >> > Jobs have been picked up, just required quite some hours to run
> >> everything.
> >> >
> >> > this morning I noticed some failed with the following message:
> >> >
> >> > org.codehaus.groovy.control.MultipleCompilationErrorsException:
> startup
> >> failed:
> >> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> >> > timestamps()
> >> > ^
> >> >
> >> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> >> > ansiColor('xterm')
> >> > ^
> >> >
> >> > 2 errors
> >> >
> >> > at
> >>
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>
> >> > at
> >> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> >> > at
> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> >> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> >> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> >> > at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > at hudson.model.Executor.run(Executor.java:429)
> >> > Finished: FAILURE
> >> >
> >> >
> >> >
> >> > I've looked into our Jenkins scripts and and can't find these
> options.
> >> So
> >> > I assume these are some globally defined scripts that needs to be
> >> repaired.
> >> >
> >> > thanks,
> >> > Robert
> >> >
> >> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
> >> > Hi
> >> >
> >> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> >> > wrote:
> >> >
> >> > > In case you haven't noticed: all Windows nodes are offline.
> >> > >
> >> > >
> >> > Yeah they are 'leased' nodes shared between all ci-* controllers and
> >> come
> >> > online
> >> > when in use.
> >> >
> >> >
> >> > Robert
> >> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
> >> > > Our scripts use the labels ubuntu and Windows to distinguish the
> OS.
> >> > > Is it safe to add ubuntu to the maven nodes?
> >> > >
> >> > > thanks,
> >> > > Robert
> >> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> >> > > Thanks All,
> >> > >
> >> > > Jobs have been migrated.
> >> > >
> >> > > Please test and report any issues ASAP to INFRA jira.
> >> > >
> >> > > https://ci-maven.apache.org/
> >> > >
> >> > >
> >> > >
> >> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> >> > > > Hi Gavin
> >> > > >
> >> > > > Let’s go. I cannot test more than the authentication today but I
> can
> >> > > assist
> >> > > > more later this month after my vacations.
> >> > > >
> >> > > > What is important is to move the shared libraries used by our
> jobs
> >> (I
> >> > do
> >> > > > not remember how they were configured).
> >> > > >
> >> > > > We will see if we need / have to find some sponsors to setup some
> >> > > > additional build nodes.
> >> > > >
> >> > > > Thanks for your help.
> >> > > >
> >> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> >> > > > écrit :
> >> > > >
> >> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> >> > > > > gmcdonald@apache.org>
> >> > > > > ha scritto:
> >> > > > >
> >> > > > > > Hi All,
> >> > > > > >
> >> > > > > > With no more interest in this subject, and no willing
> testers,
> >> > I'll
> >> > > be
> >> > > > > > going ahead tomorrow and move all Maven jobs to
> >> > ci-maven.apache.org.
> >> > > > > >
> >> > > > >
> >> > > > > I have tested that I am able to login to ci-maven with my LDAP
> >> > > credentials
> >> > > > >
> >> > > > > Thanks
> >> > > > > Enrico
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > Feel free to check after the move and create INFRA jira
> >> ticket(s)
> >> > for
> >> > > > > > anything that breaks as a result.
> >> > > > > >
> >> > > > > > Gav...
> >> > > > > >
> >> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> >> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> >> > > > > > > > > Hi Maven folks.
> >> > > > > > > > >
> >> > > > > > > > > Infra has decided to separate off the Maven build jobs
> >> from
> >> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
> >> > Controller
> >> > > and
> >> > > > > > Agents.
> >> > > > > > > > >
> >> > > > > > > > > This means that Maven now has a dedicated Jenkins
> >> > environment
> >> > > for
> >> > > > > > itself.
> >> > > > > > > > > It
> >> > > > > > > > > also means that no other projects build jobs can build
> on
> >> > the
> >> > > Maven
> >> > > > > > nodes;
> >> > > > > > > > > and
> >> > > > > > > > > then Maven jobs will no longer be able to build on the
> >> > > ci-builds
> >> > > > > > jobs.
> >> > > > > > > > >
> >> > > > > > > > > Your new Controller is set up as
> >> https://ci-maven.apache.org
> >> > > and
> >> > > > > > all Maven
> >> > > > > > > > > Committers
> >> > > > > > > > > can login via LDAP and create jobs.
> >> > > > > > > > >
> >> > > > > > > > > At the time of writing, there is one node/agent
> attached
> >> but
> >> > I
> >> > > am
> >> > > > > > building
> >> > > > > > > > > 4 more - all
> >> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
> >> > > > > > > >
> >> > > > > > > > Well, just Ubuntu? We actually need at least three
> different
> >> > > > > operating
> >> > > > > > > > systems to detect subtile bugs which I have found over
> time.
> >> > > > > > > > Since this is on an Azure can you rather add at least one
> >> > > Windows and
> >> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> >> > > > > > >
> >> > > > > > > You are getting exactly as you have been using on
> >> ci-builds.a.o
> >> > > (and
> >> > > > > > builds.a.o) before that.
> >> > > > > > > There is no FreeBSD on any of our CI nor has there been for
> >> many
> >> > > years,
> >> > > > > > we do not support
> >> > > > > > > it.
> >> > > > > > >
> >> > > > > > > As for Windows nodes, you still have access to the same
> >> Windows
> >> > > nodes
> >> > > > > as
> >> > > > > > you always have, they are what are called floating agents
> >> > available
> >> > > for
> >> > > > > > lease by all Jenkins Controllers, so your access and
> >> availability
> >> > to
> >> > > the
> >> > > > > > Windows nodes will not change. We have 6 Windoes nodes
> currently
> >> > and
> >> > > plan
> >> > > > > > to add some more soon.
> >> > > > > > >
> >> > > > > > > If anyone were to donate any nodes to attach to ci-maven
> then
> >> > we'd
> >> > > be
> >> > > > > > happy to help add them on, including FreeBSD donated nodes,
> only
> >> > > > > difference
> >> > > > > > being that Infra would not manage them nor install any
> software
> >> on
> >> > > them,
> >> > > > > > which would be up to the donator(s).
> >> > > > > > >
> >> > > > > > > HTH
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > > For additional commands, e-mail:
> dev-help@maven.apache.org
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > ---------------------------------------------------------------------
> >> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > --
> >> > > > Arnaud Héritier
> >> > > > Twitter/Skype : aheritier
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > >
> >> > >
> >> >
> >> > --
> >> >
> >> > *Gavin McDonald*
> >> > Systems Administrator
> >> > ASF Infrastructure Team
> >> >
> >> >
> >>
> >> --
> >>
> >> *Gavin McDonald*
> >> Systems Administrator
> >> ASF Infrastructure Team
> >>
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
All PMC Members should see an 'Administer' button down the leftside nav
On Fri, Jul 16, 2021 at 10:38 AM Robert Scholte <rf...@apache.org>
wrote:
> This would indeed be a good one for Arnaud.
> I don't recognize an option related to this.
> What do you expect me to see, or can you share a link to see if I have
> access?
>
> thanks,
> Robert
>
> On 16-7-2021 10:25:04, Gavin McDonald <gm...@apache.org> wrote:
> Hi
>
> I think I have it configured correctly that PMC can manage the controller,
> can someone test by
> trying to install this plugin and let me know?
>
> TIA
>
> On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier
> wrote:
>
> > This one:
> > https://plugins.jenkins.io/ws-cleanup/
> >
> > Do we (at least PMCs) have admin privileges on the controller? In such
> > case you can just add it from the plugins management admin panel.
> >
> > You just search the missing keyword on jenkins.io to find the plugin
> > implementing the missing command.
> >
> > We should probably document these needs somewhere.
> >
> > Arnaud
> >
> > Le ven. 16 juil. 2021 à 10:02, Robert Scholte a
> > écrit :
> >
> >> Yes, that fixed the build generating pages like
> >>
> >>
> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
> >>
> >>
> >> Most masters are broken, most likely all due to this error:"
> >> No such DSL method 'cleanWs'
> >>
> >>
> >> Looks like another plugin is missing.
> >>
> >> Robert
> >> On 16-7-2021 09:52:15, Gavin McDonald wrote:
> >> Thanks Robert,
> >>
> >> I believe those are a result of missing plugins Ansicolor and
> Timestamper.
> >>
> >> Both now installed
> >>
> >> HTH
> >>
> >> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
> >>
> >> > Okay, that makes sense.
> >> > Jobs have been picked up, just required quite some hours to run
> >> everything.
> >> >
> >> > this morning I noticed some failed with the following message:
> >> >
> >> > org.codehaus.groovy.control.MultipleCompilationErrorsException:
> startup
> >> failed:
> >> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> >> > timestamps()
> >> > ^
> >> >
> >> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
> >> types: [authorizationMatrix, buildDiscarder, catchError,
> >> checkoutToSubdirectory, datadog, disableConcurrentBuilds,
> disableResume,
> >> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> >> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> >> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> >> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> >> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> >> > ansiColor('xterm')
> >> > ^
> >> >
> >> > 2 errors
> >> >
> >> > at
> >>
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>
> >> > at
> >>
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>
> >> > at
> >> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> >> > at
> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> >> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> >> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>
> >> > at
> >>
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> >> > at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > at hudson.model.Executor.run(Executor.java:429)
> >> > Finished: FAILURE
> >> >
> >> >
> >> >
> >> > I've looked into our Jenkins scripts and and can't find these
> options.
> >> So
> >> > I assume these are some globally defined scripts that needs to be
> >> repaired.
> >> >
> >> > thanks,
> >> > Robert
> >> >
> >> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
> >> > Hi
> >> >
> >> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> >> > wrote:
> >> >
> >> > > In case you haven't noticed: all Windows nodes are offline.
> >> > >
> >> > >
> >> > Yeah they are 'leased' nodes shared between all ci-* controllers and
> >> come
> >> > online
> >> > when in use.
> >> >
> >> >
> >> > Robert
> >> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
> >> > > Our scripts use the labels ubuntu and Windows to distinguish the
> OS.
> >> > > Is it safe to add ubuntu to the maven nodes?
> >> > >
> >> > > thanks,
> >> > > Robert
> >> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> >> > > Thanks All,
> >> > >
> >> > > Jobs have been migrated.
> >> > >
> >> > > Please test and report any issues ASAP to INFRA jira.
> >> > >
> >> > > https://ci-maven.apache.org/
> >> > >
> >> > >
> >> > >
> >> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> >> > > > Hi Gavin
> >> > > >
> >> > > > Let’s go. I cannot test more than the authentication today but I
> can
> >> > > assist
> >> > > > more later this month after my vacations.
> >> > > >
> >> > > > What is important is to move the shared libraries used by our
> jobs
> >> (I
> >> > do
> >> > > > not remember how they were configured).
> >> > > >
> >> > > > We will see if we need / have to find some sponsors to setup some
> >> > > > additional build nodes.
> >> > > >
> >> > > > Thanks for your help.
> >> > > >
> >> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> >> > > > écrit :
> >> > > >
> >> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> >> > > > > gmcdonald@apache.org>
> >> > > > > ha scritto:
> >> > > > >
> >> > > > > > Hi All,
> >> > > > > >
> >> > > > > > With no more interest in this subject, and no willing
> testers,
> >> > I'll
> >> > > be
> >> > > > > > going ahead tomorrow and move all Maven jobs to
> >> > ci-maven.apache.org.
> >> > > > > >
> >> > > > >
> >> > > > > I have tested that I am able to login to ci-maven with my LDAP
> >> > > credentials
> >> > > > >
> >> > > > > Thanks
> >> > > > > Enrico
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > Feel free to check after the move and create INFRA jira
> >> ticket(s)
> >> > for
> >> > > > > > anything that breaks as a result.
> >> > > > > >
> >> > > > > > Gav...
> >> > > > > >
> >> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> >> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> >> > > > > > > > > Hi Maven folks.
> >> > > > > > > > >
> >> > > > > > > > > Infra has decided to separate off the Maven build jobs
> >> from
> >> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
> >> > Controller
> >> > > and
> >> > > > > > Agents.
> >> > > > > > > > >
> >> > > > > > > > > This means that Maven now has a dedicated Jenkins
> >> > environment
> >> > > for
> >> > > > > > itself.
> >> > > > > > > > > It
> >> > > > > > > > > also means that no other projects build jobs can build
> on
> >> > the
> >> > > Maven
> >> > > > > > nodes;
> >> > > > > > > > > and
> >> > > > > > > > > then Maven jobs will no longer be able to build on the
> >> > > ci-builds
> >> > > > > > jobs.
> >> > > > > > > > >
> >> > > > > > > > > Your new Controller is set up as
> >> https://ci-maven.apache.org
> >> > > and
> >> > > > > > all Maven
> >> > > > > > > > > Committers
> >> > > > > > > > > can login via LDAP and create jobs.
> >> > > > > > > > >
> >> > > > > > > > > At the time of writing, there is one node/agent
> attached
> >> but
> >> > I
> >> > > am
> >> > > > > > building
> >> > > > > > > > > 4 more - all
> >> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
> >> > > > > > > >
> >> > > > > > > > Well, just Ubuntu? We actually need at least three
> different
> >> > > > > operating
> >> > > > > > > > systems to detect subtile bugs which I have found over
> time.
> >> > > > > > > > Since this is on an Azure can you rather add at least one
> >> > > Windows and
> >> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> >> > > > > > >
> >> > > > > > > You are getting exactly as you have been using on
> >> ci-builds.a.o
> >> > > (and
> >> > > > > > builds.a.o) before that.
> >> > > > > > > There is no FreeBSD on any of our CI nor has there been for
> >> many
> >> > > years,
> >> > > > > > we do not support
> >> > > > > > > it.
> >> > > > > > >
> >> > > > > > > As for Windows nodes, you still have access to the same
> >> Windows
> >> > > nodes
> >> > > > > as
> >> > > > > > you always have, they are what are called floating agents
> >> > available
> >> > > for
> >> > > > > > lease by all Jenkins Controllers, so your access and
> >> availability
> >> > to
> >> > > the
> >> > > > > > Windows nodes will not change. We have 6 Windoes nodes
> currently
> >> > and
> >> > > plan
> >> > > > > > to add some more soon.
> >> > > > > > >
> >> > > > > > > If anyone were to donate any nodes to attach to ci-maven
> then
> >> > we'd
> >> > > be
> >> > > > > > happy to help add them on, including FreeBSD donated nodes,
> only
> >> > > > > difference
> >> > > > > > being that Infra would not manage them nor install any
> software
> >> on
> >> > > them,
> >> > > > > > which would be up to the donator(s).
> >> > > > > > >
> >> > > > > > > HTH
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > > For additional commands, e-mail:
> dev-help@maven.apache.org
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > >
> ---------------------------------------------------------------------
> >> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > ---------------------------------------------------------------------
> >> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > --
> >> > > > Arnaud Héritier
> >> > > > Twitter/Skype : aheritier
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > > For additional commands, e-mail: dev-help@maven.apache.org
> >> > >
> >> > >
> >> >
> >> > --
> >> >
> >> > *Gavin McDonald*
> >> > Systems Administrator
> >> > ASF Infrastructure Team
> >> >
> >> >
> >>
> >> --
> >>
> >> *Gavin McDonald*
> >> Systems Administrator
> >> ASF Infrastructure Team
> >>
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
This would indeed be a good one for Arnaud.
I don't recognize an option related to this.
What do you expect me to see, or can you share a link to see if I have access?
thanks,
Robert
On 16-7-2021 10:25:04, Gavin McDonald <gm...@apache.org> wrote:
Hi
I think I have it configured correctly that PMC can manage the controller,
can someone test by
trying to install this plugin and let me know?
TIA
On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier
wrote:
> This one:
> https://plugins.jenkins.io/ws-cleanup/
>
> Do we (at least PMCs) have admin privileges on the controller? In such
> case you can just add it from the plugins management admin panel.
>
> You just search the missing keyword on jenkins.io to find the plugin
> implementing the missing command.
>
> We should probably document these needs somewhere.
>
> Arnaud
>
> Le ven. 16 juil. 2021 à 10:02, Robert Scholte a
> écrit :
>
>> Yes, that fixed the build generating pages like
>>
>> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
>>
>>
>> Most masters are broken, most likely all due to this error:"
>> No such DSL method 'cleanWs'
>>
>>
>> Looks like another plugin is missing.
>>
>> Robert
>> On 16-7-2021 09:52:15, Gavin McDonald wrote:
>> Thanks Robert,
>>
>> I believe those are a result of missing plugins Ansicolor and Timestamper.
>>
>> Both now installed
>>
>> HTH
>>
>> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
>>
>> > Okay, that makes sense.
>> > Jobs have been picked up, just required quite some hours to run
>> everything.
>> >
>> > this morning I noticed some failed with the following message:
>> >
>> > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
>> failed:
>> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
>> types: [authorizationMatrix, buildDiscarder, catchError,
>> checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume,
>> durabilityHint, newContainerPerStage, overrideIndexTriggers,
>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
>> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
>> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
>> > timestamps()
>> > ^
>> >
>> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
>> types: [authorizationMatrix, buildDiscarder, catchError,
>> checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume,
>> durabilityHint, newContainerPerStage, overrideIndexTriggers,
>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
>> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
>> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
>> > ansiColor('xterm')
>> > ^
>> >
>> > 2 errors
>> >
>> > at
>> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>> > at
>> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
>> > at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
>> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
>> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>> > at
>> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
>> > at hudson.model.ResourceController.execute(ResourceController.java:97)
>> > at hudson.model.Executor.run(Executor.java:429)
>> > Finished: FAILURE
>> >
>> >
>> >
>> > I've looked into our Jenkins scripts and and can't find these options.
>> So
>> > I assume these are some globally defined scripts that needs to be
>> repaired.
>> >
>> > thanks,
>> > Robert
>> >
>> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
>> > Hi
>> >
>> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
>> > wrote:
>> >
>> > > In case you haven't noticed: all Windows nodes are offline.
>> > >
>> > >
>> > Yeah they are 'leased' nodes shared between all ci-* controllers and
>> come
>> > online
>> > when in use.
>> >
>> >
>> > Robert
>> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
>> > > Our scripts use the labels ubuntu and Windows to distinguish the OS.
>> > > Is it safe to add ubuntu to the maven nodes?
>> > >
>> > > thanks,
>> > > Robert
>> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
>> > > Thanks All,
>> > >
>> > > Jobs have been migrated.
>> > >
>> > > Please test and report any issues ASAP to INFRA jira.
>> > >
>> > > https://ci-maven.apache.org/
>> > >
>> > >
>> > >
>> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
>> > > > Hi Gavin
>> > > >
>> > > > Let’s go. I cannot test more than the authentication today but I can
>> > > assist
>> > > > more later this month after my vacations.
>> > > >
>> > > > What is important is to move the shared libraries used by our jobs
>> (I
>> > do
>> > > > not remember how they were configured).
>> > > >
>> > > > We will see if we need / have to find some sponsors to setup some
>> > > > additional build nodes.
>> > > >
>> > > > Thanks for your help.
>> > > >
>> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
>> > > > écrit :
>> > > >
>> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
>> > > > > gmcdonald@apache.org>
>> > > > > ha scritto:
>> > > > >
>> > > > > > Hi All,
>> > > > > >
>> > > > > > With no more interest in this subject, and no willing testers,
>> > I'll
>> > > be
>> > > > > > going ahead tomorrow and move all Maven jobs to
>> > ci-maven.apache.org.
>> > > > > >
>> > > > >
>> > > > > I have tested that I am able to login to ci-maven with my LDAP
>> > > credentials
>> > > > >
>> > > > > Thanks
>> > > > > Enrico
>> > > > >
>> > > > >
>> > > > > >
>> > > > > > Feel free to check after the move and create INFRA jira
>> ticket(s)
>> > for
>> > > > > > anything that breaks as a result.
>> > > > > >
>> > > > > > Gav...
>> > > > > >
>> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
>> > > > > > > Hi Michael,
>> > > > > > >
>> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
>> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
>> > > > > > > > > Hi Maven folks.
>> > > > > > > > >
>> > > > > > > > > Infra has decided to separate off the Maven build jobs
>> from
>> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
>> > Controller
>> > > and
>> > > > > > Agents.
>> > > > > > > > >
>> > > > > > > > > This means that Maven now has a dedicated Jenkins
>> > environment
>> > > for
>> > > > > > itself.
>> > > > > > > > > It
>> > > > > > > > > also means that no other projects build jobs can build on
>> > the
>> > > Maven
>> > > > > > nodes;
>> > > > > > > > > and
>> > > > > > > > > then Maven jobs will no longer be able to build on the
>> > > ci-builds
>> > > > > > jobs.
>> > > > > > > > >
>> > > > > > > > > Your new Controller is set up as
>> https://ci-maven.apache.org
>> > > and
>> > > > > > all Maven
>> > > > > > > > > Committers
>> > > > > > > > > can login via LDAP and create jobs.
>> > > > > > > > >
>> > > > > > > > > At the time of writing, there is one node/agent attached
>> but
>> > I
>> > > am
>> > > > > > building
>> > > > > > > > > 4 more - all
>> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
>> > > > > > > >
>> > > > > > > > Well, just Ubuntu? We actually need at least three different
>> > > > > operating
>> > > > > > > > systems to detect subtile bugs which I have found over time.
>> > > > > > > > Since this is on an Azure can you rather add at least one
>> > > Windows and
>> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
>> > > > > > >
>> > > > > > > You are getting exactly as you have been using on
>> ci-builds.a.o
>> > > (and
>> > > > > > builds.a.o) before that.
>> > > > > > > There is no FreeBSD on any of our CI nor has there been for
>> many
>> > > years,
>> > > > > > we do not support
>> > > > > > > it.
>> > > > > > >
>> > > > > > > As for Windows nodes, you still have access to the same
>> Windows
>> > > nodes
>> > > > > as
>> > > > > > you always have, they are what are called floating agents
>> > available
>> > > for
>> > > > > > lease by all Jenkins Controllers, so your access and
>> availability
>> > to
>> > > the
>> > > > > > Windows nodes will not change. We have 6 Windoes nodes currently
>> > and
>> > > plan
>> > > > > > to add some more soon.
>> > > > > > >
>> > > > > > > If anyone were to donate any nodes to attach to ci-maven then
>> > we'd
>> > > be
>> > > > > > happy to help add them on, including FreeBSD donated nodes, only
>> > > > > difference
>> > > > > > being that Infra would not manage them nor install any software
>> on
>> > > them,
>> > > > > > which would be up to the donator(s).
>> > > > > > >
>> > > > > > > HTH
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Michael
>> > > > > > > >
>> > > > > > > >
>> > > ---------------------------------------------------------------------
>> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > ---------------------------------------------------------------------
>> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > Arnaud Héritier
>> > > > Twitter/Skype : aheritier
>> > > >
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > For additional commands, e-mail: dev-help@maven.apache.org
>> > >
>> > >
>> >
>> > --
>> >
>> > *Gavin McDonald*
>> > Systems Administrator
>> > ASF Infrastructure Team
>> >
>> >
>>
>> --
>>
>> *Gavin McDonald*
>> Systems Administrator
>> ASF Infrastructure Team
>>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi
I think I have it configured correctly that PMC can manage the controller,
can someone test by
trying to install this plugin and let me know?
TIA
On Fri, Jul 16, 2021 at 10:07 AM Arnaud Héritier <ah...@gmail.com>
wrote:
> This one:
> https://plugins.jenkins.io/ws-cleanup/
>
> Do we (at least PMCs) have admin privileges on the controller? In such
> case you can just add it from the plugins management admin panel.
>
> You just search the missing keyword on jenkins.io to find the plugin
> implementing the missing command.
>
> We should probably document these needs somewhere.
>
> Arnaud
>
> Le ven. 16 juil. 2021 à 10:02, Robert Scholte <rf...@apache.org> a
> écrit :
>
>> Yes, that fixed the build generating pages like
>>
>> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
>>
>>
>> Most masters are broken, most likely all due to this error:"
>> No such DSL method 'cleanWs'
>>
>>
>> Looks like another plugin is missing.
>>
>> Robert
>> On 16-7-2021 09:52:15, Gavin McDonald <gm...@apache.org> wrote:
>> Thanks Robert,
>>
>> I believe those are a result of missing plugins Ansicolor and Timestamper.
>>
>> Both now installed
>>
>> HTH
>>
>> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
>>
>> > Okay, that makes sense.
>> > Jobs have been picked up, just required quite some hours to run
>> everything.
>> >
>> > this morning I noticed some failed with the following message:
>> >
>> > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
>> failed:
>> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
>> types: [authorizationMatrix, buildDiscarder, catchError,
>> checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume,
>> durabilityHint, newContainerPerStage, overrideIndexTriggers,
>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
>> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
>> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
>> > timestamps()
>> > ^
>> >
>> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option
>> types: [authorizationMatrix, buildDiscarder, catchError,
>> checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume,
>> durabilityHint, newContainerPerStage, overrideIndexTriggers,
>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
>> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
>> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
>> > ansiColor('xterm')
>> > ^
>> >
>> > 2 errors
>> >
>> > at
>> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
>> > at
>> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
>> > at
>> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
>> > at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
>> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
>> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
>> > at
>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
>> > at
>> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
>> > at hudson.model.ResourceController.execute(ResourceController.java:97)
>> > at hudson.model.Executor.run(Executor.java:429)
>> > Finished: FAILURE
>> >
>> >
>> >
>> > I've looked into our Jenkins scripts and and can't find these options.
>> So
>> > I assume these are some globally defined scripts that needs to be
>> repaired.
>> >
>> > thanks,
>> > Robert
>> >
>> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
>> > Hi
>> >
>> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
>> > wrote:
>> >
>> > > In case you haven't noticed: all Windows nodes are offline.
>> > >
>> > >
>> > Yeah they are 'leased' nodes shared between all ci-* controllers and
>> come
>> > online
>> > when in use.
>> >
>> >
>> > Robert
>> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
>> > > Our scripts use the labels ubuntu and Windows to distinguish the OS.
>> > > Is it safe to add ubuntu to the maven nodes?
>> > >
>> > > thanks,
>> > > Robert
>> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
>> > > Thanks All,
>> > >
>> > > Jobs have been migrated.
>> > >
>> > > Please test and report any issues ASAP to INFRA jira.
>> > >
>> > > https://ci-maven.apache.org/
>> > >
>> > >
>> > >
>> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
>> > > > Hi Gavin
>> > > >
>> > > > Let’s go. I cannot test more than the authentication today but I can
>> > > assist
>> > > > more later this month after my vacations.
>> > > >
>> > > > What is important is to move the shared libraries used by our jobs
>> (I
>> > do
>> > > > not remember how they were configured).
>> > > >
>> > > > We will see if we need / have to find some sponsors to setup some
>> > > > additional build nodes.
>> > > >
>> > > > Thanks for your help.
>> > > >
>> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
>> > > > écrit :
>> > > >
>> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
>> > > > > gmcdonald@apache.org>
>> > > > > ha scritto:
>> > > > >
>> > > > > > Hi All,
>> > > > > >
>> > > > > > With no more interest in this subject, and no willing testers,
>> > I'll
>> > > be
>> > > > > > going ahead tomorrow and move all Maven jobs to
>> > ci-maven.apache.org.
>> > > > > >
>> > > > >
>> > > > > I have tested that I am able to login to ci-maven with my LDAP
>> > > credentials
>> > > > >
>> > > > > Thanks
>> > > > > Enrico
>> > > > >
>> > > > >
>> > > > > >
>> > > > > > Feel free to check after the move and create INFRA jira
>> ticket(s)
>> > for
>> > > > > > anything that breaks as a result.
>> > > > > >
>> > > > > > Gav...
>> > > > > >
>> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
>> > > > > > > Hi Michael,
>> > > > > > >
>> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
>> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
>> > > > > > > > > Hi Maven folks.
>> > > > > > > > >
>> > > > > > > > > Infra has decided to separate off the Maven build jobs
>> from
>> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
>> > Controller
>> > > and
>> > > > > > Agents.
>> > > > > > > > >
>> > > > > > > > > This means that Maven now has a dedicated Jenkins
>> > environment
>> > > for
>> > > > > > itself.
>> > > > > > > > > It
>> > > > > > > > > also means that no other projects build jobs can build on
>> > the
>> > > Maven
>> > > > > > nodes;
>> > > > > > > > > and
>> > > > > > > > > then Maven jobs will no longer be able to build on the
>> > > ci-builds
>> > > > > > jobs.
>> > > > > > > > >
>> > > > > > > > > Your new Controller is set up as
>> https://ci-maven.apache.org
>> > > and
>> > > > > > all Maven
>> > > > > > > > > Committers
>> > > > > > > > > can login via LDAP and create jobs.
>> > > > > > > > >
>> > > > > > > > > At the time of writing, there is one node/agent attached
>> but
>> > I
>> > > am
>> > > > > > building
>> > > > > > > > > 4 more - all
>> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
>> > > > > > > >
>> > > > > > > > Well, just Ubuntu? We actually need at least three different
>> > > > > operating
>> > > > > > > > systems to detect subtile bugs which I have found over time.
>> > > > > > > > Since this is on an Azure can you rather add at least one
>> > > Windows and
>> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
>> > > > > > >
>> > > > > > > You are getting exactly as you have been using on
>> ci-builds.a.o
>> > > (and
>> > > > > > builds.a.o) before that.
>> > > > > > > There is no FreeBSD on any of our CI nor has there been for
>> many
>> > > years,
>> > > > > > we do not support
>> > > > > > > it.
>> > > > > > >
>> > > > > > > As for Windows nodes, you still have access to the same
>> Windows
>> > > nodes
>> > > > > as
>> > > > > > you always have, they are what are called floating agents
>> > available
>> > > for
>> > > > > > lease by all Jenkins Controllers, so your access and
>> availability
>> > to
>> > > the
>> > > > > > Windows nodes will not change. We have 6 Windoes nodes currently
>> > and
>> > > plan
>> > > > > > to add some more soon.
>> > > > > > >
>> > > > > > > If anyone were to donate any nodes to attach to ci-maven then
>> > we'd
>> > > be
>> > > > > > happy to help add them on, including FreeBSD donated nodes, only
>> > > > > difference
>> > > > > > being that Infra would not manage them nor install any software
>> on
>> > > them,
>> > > > > > which would be up to the donator(s).
>> > > > > > >
>> > > > > > > HTH
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Michael
>> > > > > > > >
>> > > > > > > >
>> > > ---------------------------------------------------------------------
>> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > ---------------------------------------------------------------------
>> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
>> > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > Arnaud Héritier
>> > > > Twitter/Skype : aheritier
>> > > >
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > > For additional commands, e-mail: dev-help@maven.apache.org
>> > >
>> > >
>> >
>> > --
>> >
>> > *Gavin McDonald*
>> > Systems Administrator
>> > ASF Infrastructure Team
>> >
>> >
>>
>> --
>>
>> *Gavin McDonald*
>> Systems Administrator
>> ASF Infrastructure Team
>>
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
This one:
https://plugins.jenkins.io/ws-cleanup/
Do we (at least PMCs) have admin privileges on the controller? In such case
you can just add it from the plugins management admin panel.
You just search the missing keyword on jenkins.io to find the plugin
implementing the missing command.
We should probably document these needs somewhere.
Arnaud
Le ven. 16 juil. 2021 à 10:02, Robert Scholte <rf...@apache.org> a
écrit :
> Yes, that fixed the build generating pages like
>
> https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
>
>
> Most masters are broken, most likely all due to this error:"
> No such DSL method 'cleanWs'
>
>
> Looks like another plugin is missing.
>
> Robert
> On 16-7-2021 09:52:15, Gavin McDonald <gm...@apache.org> wrote:
> Thanks Robert,
>
> I believe those are a result of missing plugins Ansicolor and Timestamper.
>
> Both now installed
>
> HTH
>
> On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
>
> > Okay, that makes sense.
> > Jobs have been picked up, just required quite some hours to run
> everything.
> >
> > this morning I noticed some failed with the following message:
> >
> > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
> failed:
> > WorkflowScript: 54: Invalid option type "timestamps". Valid option
> types: [authorizationMatrix, buildDiscarder, catchError,
> checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume,
> durabilityHint, newContainerPerStage, overrideIndexTriggers,
> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> > timestamps()
> > ^
> >
> > WorkflowScript: 56: Invalid option type "ansiColor". Valid option types:
> [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory,
> datadog, disableConcurrentBuilds, disableResume, durabilityHint,
> newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast,
> preserveStashes, quietPeriod, rateLimitBuilds, retry, script,
> skipDefaultCheckout, skipStagesAfterUnstable, throttle,
> throttleJobProperty, timeout, waitUntil, warnError, withChecks,
> withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> > ansiColor('xterm')
> > ^
> >
> > 2 errors
> >
> > at
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
> > at
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
> > at
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
> > at
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
> > at
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
> > at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> > at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> > at
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
> > at
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
> > at
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
> > at
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
> > at
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> > at hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Finished: FAILURE
> >
> >
> >
> > I've looked into our Jenkins scripts and and can't find these options. So
> > I assume these are some globally defined scripts that needs to be
> repaired.
> >
> > thanks,
> > Robert
> >
> > On 15-7-2021 11:18:34, Gavin McDonald wrote:
> > Hi
> >
> > On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> > wrote:
> >
> > > In case you haven't noticed: all Windows nodes are offline.
> > >
> > >
> > Yeah they are 'leased' nodes shared between all ci-* controllers and come
> > online
> > when in use.
> >
> >
> > Robert
> > > On 15-7-2021 10:29:31, Robert Scholte wrote:
> > > Our scripts use the labels ubuntu and Windows to distinguish the OS.
> > > Is it safe to add ubuntu to the maven nodes?
> > >
> > > thanks,
> > > Robert
> > > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> > > Thanks All,
> > >
> > > Jobs have been migrated.
> > >
> > > Please test and report any issues ASAP to INFRA jira.
> > >
> > > https://ci-maven.apache.org/
> > >
> > >
> > >
> > > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > > > Hi Gavin
> > > >
> > > > Let’s go. I cannot test more than the authentication today but I can
> > > assist
> > > > more later this month after my vacations.
> > > >
> > > > What is important is to move the shared libraries used by our jobs (I
> > do
> > > > not remember how they were configured).
> > > >
> > > > We will see if we need / have to find some sponsors to setup some
> > > > additional build nodes.
> > > >
> > > > Thanks for your help.
> > > >
> > > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > > > écrit :
> > > >
> > > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > > > gmcdonald@apache.org>
> > > > > ha scritto:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > With no more interest in this subject, and no willing testers,
> > I'll
> > > be
> > > > > > going ahead tomorrow and move all Maven jobs to
> > ci-maven.apache.org.
> > > > > >
> > > > >
> > > > > I have tested that I am able to login to ci-maven with my LDAP
> > > credentials
> > > > >
> > > > > Thanks
> > > > > Enrico
> > > > >
> > > > >
> > > > > >
> > > > > > Feel free to check after the move and create INFRA jira ticket(s)
> > for
> > > > > > anything that breaks as a result.
> > > > > >
> > > > > > Gav...
> > > > > >
> > > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > > > Hi Michael,
> > > > > > >
> > > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > > > Hi Maven folks.
> > > > > > > > >
> > > > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > > > ci-builds.apache.org over to its very own Jenkins
> > Controller
> > > and
> > > > > > Agents.
> > > > > > > > >
> > > > > > > > > This means that Maven now has a dedicated Jenkins
> > environment
> > > for
> > > > > > itself.
> > > > > > > > > It
> > > > > > > > > also means that no other projects build jobs can build on
> > the
> > > Maven
> > > > > > nodes;
> > > > > > > > > and
> > > > > > > > > then Maven jobs will no longer be able to build on the
> > > ci-builds
> > > > > > jobs.
> > > > > > > > >
> > > > > > > > > Your new Controller is set up as
> https://ci-maven.apache.org
> > > and
> > > > > > all Maven
> > > > > > > > > Committers
> > > > > > > > > can login via LDAP and create jobs.
> > > > > > > > >
> > > > > > > > > At the time of writing, there is one node/agent attached
> but
> > I
> > > am
> > > > > > building
> > > > > > > > > 4 more - all
> > > > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > > > >
> > > > > > > > Well, just Ubuntu? We actually need at least three different
> > > > > operating
> > > > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > > > Since this is on an Azure can you rather add at least one
> > > Windows and
> > > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > > > >
> > > > > > > You are getting exactly as you have been using on ci-builds.a.o
> > > (and
> > > > > > builds.a.o) before that.
> > > > > > > There is no FreeBSD on any of our CI nor has there been for
> many
> > > years,
> > > > > > we do not support
> > > > > > > it.
> > > > > > >
> > > > > > > As for Windows nodes, you still have access to the same Windows
> > > nodes
> > > > > as
> > > > > > you always have, they are what are called floating agents
> > available
> > > for
> > > > > > lease by all Jenkins Controllers, so your access and availability
> > to
> > > the
> > > > > > Windows nodes will not change. We have 6 Windoes nodes currently
> > and
> > > plan
> > > > > > to add some more soon.
> > > > > > >
> > > > > > > If anyone were to donate any nodes to attach to ci-maven then
> > we'd
> > > be
> > > > > > happy to help add them on, including FreeBSD donated nodes, only
> > > > > difference
> > > > > > being that Infra would not manage them nor install any software
> on
> > > them,
> > > > > > which would be up to the donator(s).
> > > > > > >
> > > > > > > HTH
> > > > > > >
> > > > > > > >
> > > > > > > > Michael
> > > > > > > >
> > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > --
> > > > Arnaud Héritier
> > > > Twitter/Skype : aheritier
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
> >
> >
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
Yes, that fixed the build generating pages like
https://ci-maven.apache.org/job/Maven/job/dist-tool-plugin/job/master/site/dist-tool-master-jobs.html
Most masters are broken, most likely all due to this error:"
No such DSL method 'cleanWs'
Looks like another plugin is missing.
Robert
On 16-7-2021 09:52:15, Gavin McDonald <gm...@apache.org> wrote:
Thanks Robert,
I believe those are a result of missing plugins Ansicolor and Timestamper.
Both now installed
HTH
On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte wrote:
> Okay, that makes sense.
> Jobs have been picked up, just required quite some hours to run everything.
>
> this morning I noticed some failed with the following message:
>
> org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
> WorkflowScript: 54: Invalid option type "timestamps". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> timestamps()
> ^
>
> WorkflowScript: 56: Invalid option type "ansiColor". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> ansiColor('xterm')
> ^
>
> 2 errors
>
> at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
> at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
> at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
> at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
> at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
> at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
> at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
> at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
> at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
> at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Finished: FAILURE
>
>
>
> I've looked into our Jenkins scripts and and can't find these options. So
> I assume these are some globally defined scripts that needs to be repaired.
>
> thanks,
> Robert
>
> On 15-7-2021 11:18:34, Gavin McDonald wrote:
> Hi
>
> On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> wrote:
>
> > In case you haven't noticed: all Windows nodes are offline.
> >
> >
> Yeah they are 'leased' nodes shared between all ci-* controllers and come
> online
> when in use.
>
>
> Robert
> > On 15-7-2021 10:29:31, Robert Scholte wrote:
> > Our scripts use the labels ubuntu and Windows to distinguish the OS.
> > Is it safe to add ubuntu to the maven nodes?
> >
> > thanks,
> > Robert
> > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> > Thanks All,
> >
> > Jobs have been migrated.
> >
> > Please test and report any issues ASAP to INFRA jira.
> >
> > https://ci-maven.apache.org/
> >
> >
> >
> > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > > Hi Gavin
> > >
> > > Let’s go. I cannot test more than the authentication today but I can
> > assist
> > > more later this month after my vacations.
> > >
> > > What is important is to move the shared libraries used by our jobs (I
> do
> > > not remember how they were configured).
> > >
> > > We will see if we need / have to find some sponsors to setup some
> > > additional build nodes.
> > >
> > > Thanks for your help.
> > >
> > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > > écrit :
> > >
> > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > > gmcdonald@apache.org>
> > > > ha scritto:
> > > >
> > > > > Hi All,
> > > > >
> > > > > With no more interest in this subject, and no willing testers,
> I'll
> > be
> > > > > going ahead tomorrow and move all Maven jobs to
> ci-maven.apache.org.
> > > > >
> > > >
> > > > I have tested that I am able to login to ci-maven with my LDAP
> > credentials
> > > >
> > > > Thanks
> > > > Enrico
> > > >
> > > >
> > > > >
> > > > > Feel free to check after the move and create INFRA jira ticket(s)
> for
> > > > > anything that breaks as a result.
> > > > >
> > > > > Gav...
> > > > >
> > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > > Hi Michael,
> > > > > >
> > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > > Hi Maven folks.
> > > > > > > >
> > > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > > ci-builds.apache.org over to its very own Jenkins
> Controller
> > and
> > > > > Agents.
> > > > > > > >
> > > > > > > > This means that Maven now has a dedicated Jenkins
> environment
> > for
> > > > > itself.
> > > > > > > > It
> > > > > > > > also means that no other projects build jobs can build on
> the
> > Maven
> > > > > nodes;
> > > > > > > > and
> > > > > > > > then Maven jobs will no longer be able to build on the
> > ci-builds
> > > > > jobs.
> > > > > > > >
> > > > > > > > Your new Controller is set up as https://ci-maven.apache.org
> > and
> > > > > all Maven
> > > > > > > > Committers
> > > > > > > > can login via LDAP and create jobs.
> > > > > > > >
> > > > > > > > At the time of writing, there is one node/agent attached but
> I
> > am
> > > > > building
> > > > > > > > 4 more - all
> > > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > > >
> > > > > > > Well, just Ubuntu? We actually need at least three different
> > > > operating
> > > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > > Since this is on an Azure can you rather add at least one
> > Windows and
> > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > > >
> > > > > > You are getting exactly as you have been using on ci-builds.a.o
> > (and
> > > > > builds.a.o) before that.
> > > > > > There is no FreeBSD on any of our CI nor has there been for many
> > years,
> > > > > we do not support
> > > > > > it.
> > > > > >
> > > > > > As for Windows nodes, you still have access to the same Windows
> > nodes
> > > > as
> > > > > you always have, they are what are called floating agents
> available
> > for
> > > > > lease by all Jenkins Controllers, so your access and availability
> to
> > the
> > > > > Windows nodes will not change. We have 6 Windoes nodes currently
> and
> > plan
> > > > > to add some more soon.
> > > > > >
> > > > > > If anyone were to donate any nodes to attach to ci-maven then
> we'd
> > be
> > > > > happy to help add them on, including FreeBSD donated nodes, only
> > > > difference
> > > > > being that Infra would not manage them nor install any software on
> > them,
> > > > > which would be up to the donator(s).
> > > > > >
> > > > > > HTH
> > > > > >
> > > > > > >
> > > > > > > Michael
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > --
> > > Arnaud Héritier
> > > Twitter/Skype : aheritier
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Thanks Robert,
I believe those are a result of missing plugins Ansicolor and Timestamper.
Both now installed
HTH
On Fri, Jul 16, 2021 at 9:31 AM Robert Scholte <rf...@apache.org> wrote:
> Okay, that makes sense.
> Jobs have been picked up, just required quite some hours to run everything.
>
> this morning I noticed some failed with the following message:
>
> org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
> WorkflowScript: 54: Invalid option type "timestamps". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9.
> timestamps()
> ^
>
> WorkflowScript: 56: Invalid option type "ansiColor". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9.
> ansiColor('xterm')
> ^
>
> 2 errors
>
> at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
> at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
> at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
> at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
> at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
> at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
> at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
> at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
> at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
> at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Finished: FAILURE
>
>
>
> I've looked into our Jenkins scripts and and can't find these options. So
> I assume these are some globally defined scripts that needs to be repaired.
>
> thanks,
> Robert
>
> On 15-7-2021 11:18:34, Gavin McDonald <gm...@apache.org> wrote:
> Hi
>
> On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
> wrote:
>
> > In case you haven't noticed: all Windows nodes are offline.
> >
> >
> Yeah they are 'leased' nodes shared between all ci-* controllers and come
> online
> when in use.
>
>
> Robert
> > On 15-7-2021 10:29:31, Robert Scholte wrote:
> > Our scripts use the labels ubuntu and Windows to distinguish the OS.
> > Is it safe to add ubuntu to the maven nodes?
> >
> > thanks,
> > Robert
> > On 15-7-2021 10:06:37, Gavin McDonald wrote:
> > Thanks All,
> >
> > Jobs have been migrated.
> >
> > Please test and report any issues ASAP to INFRA jira.
> >
> > https://ci-maven.apache.org/
> >
> >
> >
> > On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > > Hi Gavin
> > >
> > > Let’s go. I cannot test more than the authentication today but I can
> > assist
> > > more later this month after my vacations.
> > >
> > > What is important is to move the shared libraries used by our jobs (I
> do
> > > not remember how they were configured).
> > >
> > > We will see if we need / have to find some sponsors to setup some
> > > additional build nodes.
> > >
> > > Thanks for your help.
> > >
> > > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > > écrit :
> > >
> > > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > > gmcdonald@apache.org>
> > > > ha scritto:
> > > >
> > > > > Hi All,
> > > > >
> > > > > With no more interest in this subject, and no willing testers,
> I'll
> > be
> > > > > going ahead tomorrow and move all Maven jobs to
> ci-maven.apache.org.
> > > > >
> > > >
> > > > I have tested that I am able to login to ci-maven with my LDAP
> > credentials
> > > >
> > > > Thanks
> > > > Enrico
> > > >
> > > >
> > > > >
> > > > > Feel free to check after the move and create INFRA jira ticket(s)
> for
> > > > > anything that breaks as a result.
> > > > >
> > > > > Gav...
> > > > >
> > > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > > Hi Michael,
> > > > > >
> > > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > > Hi Maven folks.
> > > > > > > >
> > > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > > ci-builds.apache.org over to its very own Jenkins
> Controller
> > and
> > > > > Agents.
> > > > > > > >
> > > > > > > > This means that Maven now has a dedicated Jenkins
> environment
> > for
> > > > > itself.
> > > > > > > > It
> > > > > > > > also means that no other projects build jobs can build on
> the
> > Maven
> > > > > nodes;
> > > > > > > > and
> > > > > > > > then Maven jobs will no longer be able to build on the
> > ci-builds
> > > > > jobs.
> > > > > > > >
> > > > > > > > Your new Controller is set up as https://ci-maven.apache.org
> > and
> > > > > all Maven
> > > > > > > > Committers
> > > > > > > > can login via LDAP and create jobs.
> > > > > > > >
> > > > > > > > At the time of writing, there is one node/agent attached but
> I
> > am
> > > > > building
> > > > > > > > 4 more - all
> > > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > > >
> > > > > > > Well, just Ubuntu? We actually need at least three different
> > > > operating
> > > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > > Since this is on an Azure can you rather add at least one
> > Windows and
> > > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > > >
> > > > > > You are getting exactly as you have been using on ci-builds.a.o
> > (and
> > > > > builds.a.o) before that.
> > > > > > There is no FreeBSD on any of our CI nor has there been for many
> > years,
> > > > > we do not support
> > > > > > it.
> > > > > >
> > > > > > As for Windows nodes, you still have access to the same Windows
> > nodes
> > > > as
> > > > > you always have, they are what are called floating agents
> available
> > for
> > > > > lease by all Jenkins Controllers, so your access and availability
> to
> > the
> > > > > Windows nodes will not change. We have 6 Windoes nodes currently
> and
> > plan
> > > > > to add some more soon.
> > > > > >
> > > > > > If anyone were to donate any nodes to attach to ci-maven then
> we'd
> > be
> > > > > happy to help add them on, including FreeBSD donated nodes, only
> > > > difference
> > > > > being that Infra would not manage them nor install any software on
> > them,
> > > > > which would be up to the donator(s).
> > > > > >
> > > > > > HTH
> > > > > >
> > > > > > >
> > > > > > > Michael
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > --
> > > Arnaud Héritier
> > > Twitter/Skype : aheritier
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
Okay, that makes sense.
Jobs have been picked up, just required quite some hours to run everything.
this morning I noticed some failed with the following message:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 54: Invalid option type "timestamps". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 54, column 9. timestamps() ^ WorkflowScript: 56: Invalid option type "ansiColor". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, datadog, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, throttle, throttleJobProperty, timeout, waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, wrap, ws] @ line 56, column 9. ansiColor('xterm') ^ 2 errors at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) at groovy.lang.GroovyShell.parse(GroovyShell.java:700) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:337) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE
I've looked into our Jenkins scripts and and can't find these options. So I assume these are some globally defined scripts that needs to be repaired.
thanks,
Robert
On 15-7-2021 11:18:34, Gavin McDonald <gm...@apache.org> wrote:
Hi
On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte
wrote:
> In case you haven't noticed: all Windows nodes are offline.
>
>
Yeah they are 'leased' nodes shared between all ci-* controllers and come
online
when in use.
Robert
> On 15-7-2021 10:29:31, Robert Scholte wrote:
> Our scripts use the labels ubuntu and Windows to distinguish the OS.
> Is it safe to add ubuntu to the maven nodes?
>
> thanks,
> Robert
> On 15-7-2021 10:06:37, Gavin McDonald wrote:
> Thanks All,
>
> Jobs have been migrated.
>
> Please test and report any issues ASAP to INFRA jira.
>
> https://ci-maven.apache.org/
>
>
>
> On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > Hi Gavin
> >
> > Let’s go. I cannot test more than the authentication today but I can
> assist
> > more later this month after my vacations.
> >
> > What is important is to move the shared libraries used by our jobs (I do
> > not remember how they were configured).
> >
> > We will see if we need / have to find some sponsors to setup some
> > additional build nodes.
> >
> > Thanks for your help.
> >
> > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > écrit :
> >
> > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > gmcdonald@apache.org>
> > > ha scritto:
> > >
> > > > Hi All,
> > > >
> > > > With no more interest in this subject, and no willing testers, I'll
> be
> > > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > > >
> > >
> > > I have tested that I am able to login to ci-maven with my LDAP
> credentials
> > >
> > > Thanks
> > > Enrico
> > >
> > >
> > > >
> > > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > > anything that breaks as a result.
> > > >
> > > > Gav...
> > > >
> > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > Hi Michael,
> > > > >
> > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > Hi Maven folks.
> > > > > > >
> > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > ci-builds.apache.org over to its very own Jenkins Controller
> and
> > > > Agents.
> > > > > > >
> > > > > > > This means that Maven now has a dedicated Jenkins environment
> for
> > > > itself.
> > > > > > > It
> > > > > > > also means that no other projects build jobs can build on the
> Maven
> > > > nodes;
> > > > > > > and
> > > > > > > then Maven jobs will no longer be able to build on the
> ci-builds
> > > > jobs.
> > > > > > >
> > > > > > > Your new Controller is set up as https://ci-maven.apache.org
> and
> > > > all Maven
> > > > > > > Committers
> > > > > > > can login via LDAP and create jobs.
> > > > > > >
> > > > > > > At the time of writing, there is one node/agent attached but I
> am
> > > > building
> > > > > > > 4 more - all
> > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > >
> > > > > > Well, just Ubuntu? We actually need at least three different
> > > operating
> > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > Since this is on an Azure can you rather add at least one
> Windows and
> > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > >
> > > > > You are getting exactly as you have been using on ci-builds.a.o
> (and
> > > > builds.a.o) before that.
> > > > > There is no FreeBSD on any of our CI nor has there been for many
> years,
> > > > we do not support
> > > > > it.
> > > > >
> > > > > As for Windows nodes, you still have access to the same Windows
> nodes
> > > as
> > > > you always have, they are what are called floating agents available
> for
> > > > lease by all Jenkins Controllers, so your access and availability to
> the
> > > > Windows nodes will not change. We have 6 Windoes nodes currently and
> plan
> > > > to add some more soon.
> > > > >
> > > > > If anyone were to donate any nodes to attach to ci-maven then we'd
> be
> > > > happy to help add them on, including FreeBSD donated nodes, only
> > > difference
> > > > being that Infra would not manage them nor install any software on
> them,
> > > > which would be up to the donator(s).
> > > > >
> > > > > HTH
> > > > >
> > > > > >
> > > > > > Michael
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi
On Thu, Jul 15, 2021 at 11:08 AM Robert Scholte <rf...@apache.org>
wrote:
> In case you haven't noticed: all Windows nodes are offline.
>
>
Yeah they are 'leased' nodes shared between all ci-* controllers and come
online
when in use.
Robert
> On 15-7-2021 10:29:31, Robert Scholte <rf...@apache.org> wrote:
> Our scripts use the labels ubuntu and Windows to distinguish the OS.
> Is it safe to add ubuntu to the maven nodes?
>
> thanks,
> Robert
> On 15-7-2021 10:06:37, Gavin McDonald <gm...@apache.org> wrote:
> Thanks All,
>
> Jobs have been migrated.
>
> Please test and report any issues ASAP to INFRA jira.
>
> https://ci-maven.apache.org/
>
>
>
> On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> > Hi Gavin
> >
> > Let’s go. I cannot test more than the authentication today but I can
> assist
> > more later this month after my vacations.
> >
> > What is important is to move the shared libraries used by our jobs (I do
> > not remember how they were configured).
> >
> > We will see if we need / have to find some sponsors to setup some
> > additional build nodes.
> >
> > Thanks for your help.
> >
> > Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> > écrit :
> >
> > > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > > gmcdonald@apache.org>
> > > ha scritto:
> > >
> > > > Hi All,
> > > >
> > > > With no more interest in this subject, and no willing testers, I'll
> be
> > > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > > >
> > >
> > > I have tested that I am able to login to ci-maven with my LDAP
> credentials
> > >
> > > Thanks
> > > Enrico
> > >
> > >
> > > >
> > > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > > anything that breaks as a result.
> > > >
> > > > Gav...
> > > >
> > > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > > Hi Michael,
> > > > >
> > > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > > Hi Maven folks.
> > > > > > >
> > > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > > ci-builds.apache.org over to its very own Jenkins Controller
> and
> > > > Agents.
> > > > > > >
> > > > > > > This means that Maven now has a dedicated Jenkins environment
> for
> > > > itself.
> > > > > > > It
> > > > > > > also means that no other projects build jobs can build on the
> Maven
> > > > nodes;
> > > > > > > and
> > > > > > > then Maven jobs will no longer be able to build on the
> ci-builds
> > > > jobs.
> > > > > > >
> > > > > > > Your new Controller is set up as https://ci-maven.apache.org
> and
> > > > all Maven
> > > > > > > Committers
> > > > > > > can login via LDAP and create jobs.
> > > > > > >
> > > > > > > At the time of writing, there is one node/agent attached but I
> am
> > > > building
> > > > > > > 4 more - all
> > > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > > >
> > > > > > Well, just Ubuntu? We actually need at least three different
> > > operating
> > > > > > systems to detect subtile bugs which I have found over time.
> > > > > > Since this is on an Azure can you rather add at least one
> Windows and
> > > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > > >
> > > > > You are getting exactly as you have been using on ci-builds.a.o
> (and
> > > > builds.a.o) before that.
> > > > > There is no FreeBSD on any of our CI nor has there been for many
> years,
> > > > we do not support
> > > > > it.
> > > > >
> > > > > As for Windows nodes, you still have access to the same Windows
> nodes
> > > as
> > > > you always have, they are what are called floating agents available
> for
> > > > lease by all Jenkins Controllers, so your access and availability to
> the
> > > > Windows nodes will not change. We have 6 Windoes nodes currently and
> plan
> > > > to add some more soon.
> > > > >
> > > > > If anyone were to donate any nodes to attach to ci-maven then we'd
> be
> > > > happy to help add them on, including FreeBSD donated nodes, only
> > > difference
> > > > being that Infra would not manage them nor install any software on
> them,
> > > > which would be up to the donator(s).
> > > > >
> > > > > HTH
> > > > >
> > > > > >
> > > > > > Michael
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > --
> > Arnaud Héritier
> > Twitter/Skype : aheritier
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
In case you haven't noticed: all Windows nodes are offline.
Robert
On 15-7-2021 10:29:31, Robert Scholte <rf...@apache.org> wrote:
Our scripts use the labels ubuntu and Windows to distinguish the OS.
Is it safe to add ubuntu to the maven nodes?
thanks,
Robert
On 15-7-2021 10:06:37, Gavin McDonald <gm...@apache.org> wrote:
Thanks All,
Jobs have been migrated.
Please test and report any issues ASAP to INFRA jira.
https://ci-maven.apache.org/
On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> Hi Gavin
>
> Let’s go. I cannot test more than the authentication today but I can assist
> more later this month after my vacations.
>
> What is important is to move the shared libraries used by our jobs (I do
> not remember how they were configured).
>
> We will see if we need / have to find some sponsors to setup some
> additional build nodes.
>
> Thanks for your help.
>
> Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> écrit :
>
> > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > gmcdonald@apache.org>
> > ha scritto:
> >
> > > Hi All,
> > >
> > > With no more interest in this subject, and no willing testers, I'll be
> > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > >
> >
> > I have tested that I am able to login to ci-maven with my LDAP credentials
> >
> > Thanks
> > Enrico
> >
> >
> > >
> > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > anything that breaks as a result.
> > >
> > > Gav...
> > >
> > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > Hi Michael,
> > > >
> > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > Hi Maven folks.
> > > > > >
> > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > ci-builds.apache.org over to its very own Jenkins Controller and
> > > Agents.
> > > > > >
> > > > > > This means that Maven now has a dedicated Jenkins environment for
> > > itself.
> > > > > > It
> > > > > > also means that no other projects build jobs can build on the Maven
> > > nodes;
> > > > > > and
> > > > > > then Maven jobs will no longer be able to build on the ci-builds
> > > jobs.
> > > > > >
> > > > > > Your new Controller is set up as https://ci-maven.apache.org and
> > > all Maven
> > > > > > Committers
> > > > > > can login via LDAP and create jobs.
> > > > > >
> > > > > > At the time of writing, there is one node/agent attached but I am
> > > building
> > > > > > 4 more - all
> > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > >
> > > > > Well, just Ubuntu? We actually need at least three different
> > operating
> > > > > systems to detect subtile bugs which I have found over time.
> > > > > Since this is on an Azure can you rather add at least one Windows and
> > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > >
> > > > You are getting exactly as you have been using on ci-builds.a.o (and
> > > builds.a.o) before that.
> > > > There is no FreeBSD on any of our CI nor has there been for many years,
> > > we do not support
> > > > it.
> > > >
> > > > As for Windows nodes, you still have access to the same Windows nodes
> > as
> > > you always have, they are what are called floating agents available for
> > > lease by all Jenkins Controllers, so your access and availability to the
> > > Windows nodes will not change. We have 6 Windoes nodes currently and plan
> > > to add some more soon.
> > > >
> > > > If anyone were to donate any nodes to attach to ci-maven then we'd be
> > > happy to help add them on, including FreeBSD donated nodes, only
> > difference
> > > being that Infra would not manage them nor install any software on them,
> > > which would be up to the donator(s).
> > > >
> > > > HTH
> > > >
> > > > >
> > > > > Michael
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [JENKINS] - New Maven Controller for the project
Posted by Robert Scholte <rf...@apache.org>.
Our scripts use the labels ubuntu and Windows to distinguish the OS.
Is it safe to add ubuntu to the maven nodes?
thanks,
Robert
On 15-7-2021 10:06:37, Gavin McDonald <gm...@apache.org> wrote:
Thanks All,
Jobs have been migrated.
Please test and report any issues ASAP to INFRA jira.
https://ci-maven.apache.org/
On 2021/07/08 21:37:43, Arnaud Héritier wrote:
> Hi Gavin
>
> Let’s go. I cannot test more than the authentication today but I can assist
> more later this month after my vacations.
>
> What is important is to move the shared libraries used by our jobs (I do
> not remember how they were configured).
>
> We will see if we need / have to find some sponsors to setup some
> additional build nodes.
>
> Thanks for your help.
>
> Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli a
> écrit :
>
> > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald
> > gmcdonald@apache.org>
> > ha scritto:
> >
> > > Hi All,
> > >
> > > With no more interest in this subject, and no willing testers, I'll be
> > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > >
> >
> > I have tested that I am able to login to ci-maven with my LDAP credentials
> >
> > Thanks
> > Enrico
> >
> >
> > >
> > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > anything that breaks as a result.
> > >
> > > Gav...
> > >
> > > On 2021/06/30 18:23:54, Gavin McDonald wrote:
> > > > Hi Michael,
> > > >
> > > > On 2021/06/30 18:12:12, Michael Osipov wrote:
> > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > Hi Maven folks.
> > > > > >
> > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > ci-builds.apache.org over to its very own Jenkins Controller and
> > > Agents.
> > > > > >
> > > > > > This means that Maven now has a dedicated Jenkins environment for
> > > itself.
> > > > > > It
> > > > > > also means that no other projects build jobs can build on the Maven
> > > nodes;
> > > > > > and
> > > > > > then Maven jobs will no longer be able to build on the ci-builds
> > > jobs.
> > > > > >
> > > > > > Your new Controller is set up as https://ci-maven.apache.org and
> > > all Maven
> > > > > > Committers
> > > > > > can login via LDAP and create jobs.
> > > > > >
> > > > > > At the time of writing, there is one node/agent attached but I am
> > > building
> > > > > > 4 more - all
> > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > >
> > > > > Well, just Ubuntu? We actually need at least three different
> > operating
> > > > > systems to detect subtile bugs which I have found over time.
> > > > > Since this is on an Azure can you rather add at least one Windows and
> > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > >
> > > > You are getting exactly as you have been using on ci-builds.a.o (and
> > > builds.a.o) before that.
> > > > There is no FreeBSD on any of our CI nor has there been for many years,
> > > we do not support
> > > > it.
> > > >
> > > > As for Windows nodes, you still have access to the same Windows nodes
> > as
> > > you always have, they are what are called floating agents available for
> > > lease by all Jenkins Controllers, so your access and availability to the
> > > Windows nodes will not change. We have 6 Windoes nodes currently and plan
> > > to add some more soon.
> > > >
> > > > If anyone were to donate any nodes to attach to ci-maven then we'd be
> > > happy to help add them on, including FreeBSD donated nodes, only
> > difference
> > > being that Infra would not manage them nor install any software on them,
> > > which would be up to the donator(s).
> > > >
> > > > HTH
> > > >
> > > > >
> > > > > Michael
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Thanks All,
Jobs have been migrated.
Please test and report any issues ASAP to INFRA jira.
https://ci-maven.apache.org/
On 2021/07/08 21:37:43, Arnaud Héritier <ah...@gmail.com> wrote:
> Hi Gavin
>
> Let’s go. I cannot test more than the authentication today but I can assist
> more later this month after my vacations.
>
> What is important is to move the shared libraries used by our jobs (I do
> not remember how they were configured).
>
> We will see if we need / have to find some sponsors to setup some
> additional build nodes.
>
> Thanks for your help.
>
> Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald <
> > gmcdonald@apache.org>
> > ha scritto:
> >
> > > Hi All,
> > >
> > > With no more interest in this subject, and no willing testers, I'll be
> > > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> > >
> >
> > I have tested that I am able to login to ci-maven with my LDAP credentials
> >
> > Thanks
> > Enrico
> >
> >
> > >
> > > Feel free to check after the move and create INFRA jira ticket(s) for
> > > anything that breaks as a result.
> > >
> > > Gav...
> > >
> > > On 2021/06/30 18:23:54, Gavin McDonald <gm...@apache.org> wrote:
> > > > Hi Michael,
> > > >
> > > > On 2021/06/30 18:12:12, Michael Osipov <mi...@apache.org> wrote:
> > > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > > Hi Maven folks.
> > > > > >
> > > > > > Infra has decided to separate off the Maven build jobs from
> > > > > > ci-builds.apache.org over to its very own Jenkins Controller and
> > > Agents.
> > > > > >
> > > > > > This means that Maven now has a dedicated Jenkins environment for
> > > itself.
> > > > > > It
> > > > > > also means that no other projects build jobs can build on the Maven
> > > nodes;
> > > > > > and
> > > > > > then Maven jobs will no longer be able to build on the ci-builds
> > > jobs.
> > > > > >
> > > > > > Your new Controller is set up as https://ci-maven.apache.org and
> > > all Maven
> > > > > > Committers
> > > > > > can login via LDAP and create jobs.
> > > > > >
> > > > > > At the time of writing, there is one node/agent attached but I am
> > > building
> > > > > > 4 more - all
> > > > > > Ubuntu 20.04 and based in our Azure account.
> > > > >
> > > > > Well, just Ubuntu? We actually need at least three different
> > operating
> > > > > systems to detect subtile bugs which I have found over time.
> > > > > Since this is on an Azure can you rather add at least one Windows and
> > > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > > >
> > > > You are getting exactly as you have been using on ci-builds.a.o (and
> > > builds.a.o) before that.
> > > > There is no FreeBSD on any of our CI nor has there been for many years,
> > > we do not support
> > > > it.
> > > >
> > > > As for Windows nodes, you still have access to the same Windows nodes
> > as
> > > you always have, they are what are called floating agents available for
> > > lease by all Jenkins Controllers, so your access and availability to the
> > > Windows nodes will not change. We have 6 Windoes nodes currently and plan
> > > to add some more soon.
> > > >
> > > > If anyone were to donate any nodes to attach to ci-maven then we'd be
> > > happy to help add them on, including FreeBSD donated nodes, only
> > difference
> > > being that Infra would not manage them nor install any software on them,
> > > which would be up to the donator(s).
> > > >
> > > > HTH
> > > >
> > > > >
> > > > > Michael
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> --
> Arnaud Héritier
> Twitter/Skype : aheritier
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [JENKINS] - New Maven Controller for the project
Posted by Arnaud Héritier <ah...@gmail.com>.
Hi Gavin
Let’s go. I cannot test more than the authentication today but I can assist
more later this month after my vacations.
What is important is to move the shared libraries used by our jobs (I do
not remember how they were configured).
We will see if we need / have to find some sponsors to setup some
additional build nodes.
Thanks for your help.
Le jeu. 8 juil. 2021 à 23:06, Enrico Olivelli <eo...@gmail.com> a
écrit :
> Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald <
> gmcdonald@apache.org>
> ha scritto:
>
> > Hi All,
> >
> > With no more interest in this subject, and no willing testers, I'll be
> > going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
> >
>
> I have tested that I am able to login to ci-maven with my LDAP credentials
>
> Thanks
> Enrico
>
>
> >
> > Feel free to check after the move and create INFRA jira ticket(s) for
> > anything that breaks as a result.
> >
> > Gav...
> >
> > On 2021/06/30 18:23:54, Gavin McDonald <gm...@apache.org> wrote:
> > > Hi Michael,
> > >
> > > On 2021/06/30 18:12:12, Michael Osipov <mi...@apache.org> wrote:
> > > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > > Hi Maven folks.
> > > > >
> > > > > Infra has decided to separate off the Maven build jobs from
> > > > > ci-builds.apache.org over to its very own Jenkins Controller and
> > Agents.
> > > > >
> > > > > This means that Maven now has a dedicated Jenkins environment for
> > itself.
> > > > > It
> > > > > also means that no other projects build jobs can build on the Maven
> > nodes;
> > > > > and
> > > > > then Maven jobs will no longer be able to build on the ci-builds
> > jobs.
> > > > >
> > > > > Your new Controller is set up as https://ci-maven.apache.org and
> > all Maven
> > > > > Committers
> > > > > can login via LDAP and create jobs.
> > > > >
> > > > > At the time of writing, there is one node/agent attached but I am
> > building
> > > > > 4 more - all
> > > > > Ubuntu 20.04 and based in our Azure account.
> > > >
> > > > Well, just Ubuntu? We actually need at least three different
> operating
> > > > systems to detect subtile bugs which I have found over time.
> > > > Since this is on an Azure can you rather add at least one Windows and
> > > > FreeBSD node to it? 4x Ubuntu won't really help.
> > >
> > > You are getting exactly as you have been using on ci-builds.a.o (and
> > builds.a.o) before that.
> > > There is no FreeBSD on any of our CI nor has there been for many years,
> > we do not support
> > > it.
> > >
> > > As for Windows nodes, you still have access to the same Windows nodes
> as
> > you always have, they are what are called floating agents available for
> > lease by all Jenkins Controllers, so your access and availability to the
> > Windows nodes will not change. We have 6 Windoes nodes currently and plan
> > to add some more soon.
> > >
> > > If anyone were to donate any nodes to attach to ci-maven then we'd be
> > happy to help add them on, including FreeBSD donated nodes, only
> difference
> > being that Infra would not manage them nor install any software on them,
> > which would be up to the donator(s).
> > >
> > > HTH
> > >
> > > >
> > > > Michael
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
--
Arnaud Héritier
Twitter/Skype : aheritier
Re: [JENKINS] - New Maven Controller for the project
Posted by Enrico Olivelli <eo...@gmail.com>.
Il giorno gio 8 lug 2021 alle ore 20:06 Gavin McDonald <gm...@apache.org>
ha scritto:
> Hi All,
>
> With no more interest in this subject, and no willing testers, I'll be
> going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
>
I have tested that I am able to login to ci-maven with my LDAP credentials
Thanks
Enrico
>
> Feel free to check after the move and create INFRA jira ticket(s) for
> anything that breaks as a result.
>
> Gav...
>
> On 2021/06/30 18:23:54, Gavin McDonald <gm...@apache.org> wrote:
> > Hi Michael,
> >
> > On 2021/06/30 18:12:12, Michael Osipov <mi...@apache.org> wrote:
> > > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > > Hi Maven folks.
> > > >
> > > > Infra has decided to separate off the Maven build jobs from
> > > > ci-builds.apache.org over to its very own Jenkins Controller and
> Agents.
> > > >
> > > > This means that Maven now has a dedicated Jenkins environment for
> itself.
> > > > It
> > > > also means that no other projects build jobs can build on the Maven
> nodes;
> > > > and
> > > > then Maven jobs will no longer be able to build on the ci-builds
> jobs.
> > > >
> > > > Your new Controller is set up as https://ci-maven.apache.org and
> all Maven
> > > > Committers
> > > > can login via LDAP and create jobs.
> > > >
> > > > At the time of writing, there is one node/agent attached but I am
> building
> > > > 4 more - all
> > > > Ubuntu 20.04 and based in our Azure account.
> > >
> > > Well, just Ubuntu? We actually need at least three different operating
> > > systems to detect subtile bugs which I have found over time.
> > > Since this is on an Azure can you rather add at least one Windows and
> > > FreeBSD node to it? 4x Ubuntu won't really help.
> >
> > You are getting exactly as you have been using on ci-builds.a.o (and
> builds.a.o) before that.
> > There is no FreeBSD on any of our CI nor has there been for many years,
> we do not support
> > it.
> >
> > As for Windows nodes, you still have access to the same Windows nodes as
> you always have, they are what are called floating agents available for
> lease by all Jenkins Controllers, so your access and availability to the
> Windows nodes will not change. We have 6 Windoes nodes currently and plan
> to add some more soon.
> >
> > If anyone were to donate any nodes to attach to ci-maven then we'd be
> happy to help add them on, including FreeBSD donated nodes, only difference
> being that Infra would not manage them nor install any software on them,
> which would be up to the donator(s).
> >
> > HTH
> >
> > >
> > > Michael
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi All,
With no more interest in this subject, and no willing testers, I'll be going ahead tomorrow and move all Maven jobs to ci-maven.apache.org.
Feel free to check after the move and create INFRA jira ticket(s) for anything that breaks as a result.
Gav...
On 2021/06/30 18:23:54, Gavin McDonald <gm...@apache.org> wrote:
> Hi Michael,
>
> On 2021/06/30 18:12:12, Michael Osipov <mi...@apache.org> wrote:
> > Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > > Hi Maven folks.
> > >
> > > Infra has decided to separate off the Maven build jobs from
> > > ci-builds.apache.org over to its very own Jenkins Controller and Agents.
> > >
> > > This means that Maven now has a dedicated Jenkins environment for itself.
> > > It
> > > also means that no other projects build jobs can build on the Maven nodes;
> > > and
> > > then Maven jobs will no longer be able to build on the ci-builds jobs.
> > >
> > > Your new Controller is set up as https://ci-maven.apache.org and all Maven
> > > Committers
> > > can login via LDAP and create jobs.
> > >
> > > At the time of writing, there is one node/agent attached but I am building
> > > 4 more - all
> > > Ubuntu 20.04 and based in our Azure account.
> >
> > Well, just Ubuntu? We actually need at least three different operating
> > systems to detect subtile bugs which I have found over time.
> > Since this is on an Azure can you rather add at least one Windows and
> > FreeBSD node to it? 4x Ubuntu won't really help.
>
> You are getting exactly as you have been using on ci-builds.a.o (and builds.a.o) before that.
> There is no FreeBSD on any of our CI nor has there been for many years, we do not support
> it.
>
> As for Windows nodes, you still have access to the same Windows nodes as you always have, they are what are called floating agents available for lease by all Jenkins Controllers, so your access and availability to the Windows nodes will not change. We have 6 Windoes nodes currently and plan to add some more soon.
>
> If anyone were to donate any nodes to attach to ci-maven then we'd be happy to help add them on, including FreeBSD donated nodes, only difference being that Infra would not manage them nor install any software on them, which would be up to the donator(s).
>
> HTH
>
> >
> > Michael
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [JENKINS] - New Maven Controller for the project
Posted by Gavin McDonald <gm...@apache.org>.
Hi Michael,
On 2021/06/30 18:12:12, Michael Osipov <mi...@apache.org> wrote:
> Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> > Hi Maven folks.
> >
> > Infra has decided to separate off the Maven build jobs from
> > ci-builds.apache.org over to its very own Jenkins Controller and Agents.
> >
> > This means that Maven now has a dedicated Jenkins environment for itself.
> > It
> > also means that no other projects build jobs can build on the Maven nodes;
> > and
> > then Maven jobs will no longer be able to build on the ci-builds jobs.
> >
> > Your new Controller is set up as https://ci-maven.apache.org and all Maven
> > Committers
> > can login via LDAP and create jobs.
> >
> > At the time of writing, there is one node/agent attached but I am building
> > 4 more - all
> > Ubuntu 20.04 and based in our Azure account.
>
> Well, just Ubuntu? We actually need at least three different operating
> systems to detect subtile bugs which I have found over time.
> Since this is on an Azure can you rather add at least one Windows and
> FreeBSD node to it? 4x Ubuntu won't really help.
You are getting exactly as you have been using on ci-builds.a.o (and builds.a.o) before that.
There is no FreeBSD on any of our CI nor has there been for many years, we do not support
it.
As for Windows nodes, you still have access to the same Windows nodes as you always have, they are what are called floating agents available for lease by all Jenkins Controllers, so your access and availability to the Windows nodes will not change. We have 6 Windoes nodes currently and plan to add some more soon.
If anyone were to donate any nodes to attach to ci-maven then we'd be happy to help add them on, including FreeBSD donated nodes, only difference being that Infra would not manage them nor install any software on them, which would be up to the donator(s).
HTH
>
> Michael
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [JENKINS] - New Maven Controller for the project
Posted by Michael Osipov <mi...@apache.org>.
Am 2021-06-30 um 20:03 schrieb Gavin McDonald:
> Hi Maven folks.
>
> Infra has decided to separate off the Maven build jobs from
> ci-builds.apache.org over to its very own Jenkins Controller and Agents.
>
> This means that Maven now has a dedicated Jenkins environment for itself.
> It
> also means that no other projects build jobs can build on the Maven nodes;
> and
> then Maven jobs will no longer be able to build on the ci-builds jobs.
>
> Your new Controller is set up as https://ci-maven.apache.org and all Maven
> Committers
> can login via LDAP and create jobs.
>
> At the time of writing, there is one node/agent attached but I am building
> 4 more - all
> Ubuntu 20.04 and based in our Azure account.
Well, just Ubuntu? We actually need at least three different operating
systems to detect subtile bugs which I have found over time.
Since this is on an Azure can you rather add at least one Windows and
FreeBSD node to it? 4x Ubuntu won't really help.
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org