You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Bertrand Delacretaz <bd...@apache.org> on 2020/03/23 15:42:40 UTC
More "websites" executors needed?
Hi,
Looking at https://builds.apache.org/label/websites/load-statistics
IIUC there's only 2 executors for the "websites" label and they are
very often both busy.
We're regularly getting timeouts lately, waiting for such an executor
in the Sling website build,
https://builds.apache.org/job/Sling/job/sling-site/job/master/
Do we need more executors, or is something else wrong?
-Bertrand
Re: More "websites" executors needed?
Posted by Vladimir Sitnikov <si...@gmail.com>.
>for which we run builds.
1) You should not allow the execution of untrusted code (e.g. PR) in the
ASF infrastructure.
As far as I understand, nothing prevents putting malicious code in the
Jenkinsfile, then ASF Jenkins would execute it and reveal the passwords or
whatever.
2) Why do you need `website` executors for building PRs?
Are you updating the website based on the PR?
Vladimir
Re: More "websites" executors needed?
Posted by Gavin McDonald <ip...@gmail.com>.
On Fri, Mar 27, 2020 at 8:49 PM Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> >I've figured out it's quite easy to
> >get pull request builds and previews using Netlify.
>
> That is interesting because I raised exactly the same question 1 year ago,
> and https://issues.apache.org/jira/browse/INFRA-18102 decision was
> Netlify can't be used ASF-wide due to limitations of the free OSS tier :-/
>
It's currently not systemwide, but in use by 3 projects, we'll keep an eye
on it but
currently seems to be working ok for these 3.
Gav...
>
> Vladimir
>
--
Gav...
Re: More "websites" executors needed?
Posted by Vladimir Sitnikov <si...@gmail.com>.
>I've figured out it's quite easy to
>get pull request builds and previews using Netlify.
That is interesting because I raised exactly the same question 1 year ago,
and https://issues.apache.org/jira/browse/INFRA-18102 decision was
Netlify can't be used ASF-wide due to limitations of the free OSS tier :-/
Vladimir
Re: More "websites" executors needed?
Posted by Zoran Regvart <zo...@regvart.com>.
Hi all,
in case someone else has the need, I've figured out it's quite easy to
get pull request builds and previews using Netlify.
They have an open source plan[1] that I have applied Apache Camel to.
zoran
[1] https://www.netlify.com/legal/open-source-policy/
On Fri, Mar 27, 2020 at 4:26 PM Zoran Regvart <zo...@regvart.com> wrote:
>
> Hi builders,
> I'll disable pull request builds and look for alternatives. Anyone has
> any experience with Travis serving HTML files and can help out with
> this?
>
> zoran
>
> On Fri, Mar 27, 2020 at 3:13 PM Gavin McDonald <ip...@gmail.com> wrote:
> >
> > Camels builds are definitely an issue and are consuming the 'websites'
> > label seemingly constantly from many PRs, minor changes
> > are triggering a new complete build each time. Builds are taking a minimum
> > of 20 minutes each time.
> >
> > This needs to be looked into and see if there can be a better workflow.
> >
> > Gav...
> >
> >
> > On Thu, Mar 26, 2020 at 4:40 PM Robert Munteanu <ro...@apache.org> wrote:
> >
> > > Hello Zoran,
> > >
> > > On Thu, 2020-03-26 at 16:06 +0100, Zoran Regvart wrote:
> > > > Hi builders,
> > > > Apache Camel is participating in the Outreachy program[1], and we
> > > > have
> > > > a lot of pull requests[2] for which we run builds.
> > >
> > > That's cool and a nice problem to have :-)
> > >
> > > >
> > > > I think the other projects are having issues because of this. Not
> > > > sure
> > > > what can be done about this, we have no other way to check and
> > > > preview
> > > > the change if we don't run the build.
> > >
> > > I looked at the preview step of one of your builds and it seems that
> > > it's not actually publishing the changes anywhere, but instead
> > > archiving them in Jenkins
> > >
> > > [2020-03-26T13:19:05.101Z] [htmlpublisher] Archiving HTML reports...
> > > [2020-03-26T13:19:05.103Z] [htmlpublisher] Archiving at PROJECT level
> > > /home/jenkins/jenkins-slave/workspace/Camel.website_PR-273/camel-
> > > website/public to /x1/jenkins/jenkins-
> > > home/jobs/Camel.website/branches/PR-273/htmlreports/Preview
> > >
> > > I think that means you can run the PR check on any executor, e.g.
> > > 'ubuntu', just that the regular build needs to build on 'websites'.
> > >
> > > Note that this is independent of Vladimir's email discussing whether
> > > PRs should be built on ASF resources or not, just a comment on how to
> > > optimise executor usage.
> > >
> > > Thanks,
> > > Robert
> > >
> > >
> >
> > --
> > Gav...
>
>
>
> --
> Zoran Regvart
--
Zoran Regvart
Re: More "websites" executors needed?
Posted by Zoran Regvart <zo...@regvart.com>.
Hi builders,
I'll disable pull request builds and look for alternatives. Anyone has
any experience with Travis serving HTML files and can help out with
this?
zoran
On Fri, Mar 27, 2020 at 3:13 PM Gavin McDonald <ip...@gmail.com> wrote:
>
> Camels builds are definitely an issue and are consuming the 'websites'
> label seemingly constantly from many PRs, minor changes
> are triggering a new complete build each time. Builds are taking a minimum
> of 20 minutes each time.
>
> This needs to be looked into and see if there can be a better workflow.
>
> Gav...
>
>
> On Thu, Mar 26, 2020 at 4:40 PM Robert Munteanu <ro...@apache.org> wrote:
>
> > Hello Zoran,
> >
> > On Thu, 2020-03-26 at 16:06 +0100, Zoran Regvart wrote:
> > > Hi builders,
> > > Apache Camel is participating in the Outreachy program[1], and we
> > > have
> > > a lot of pull requests[2] for which we run builds.
> >
> > That's cool and a nice problem to have :-)
> >
> > >
> > > I think the other projects are having issues because of this. Not
> > > sure
> > > what can be done about this, we have no other way to check and
> > > preview
> > > the change if we don't run the build.
> >
> > I looked at the preview step of one of your builds and it seems that
> > it's not actually publishing the changes anywhere, but instead
> > archiving them in Jenkins
> >
> > [2020-03-26T13:19:05.101Z] [htmlpublisher] Archiving HTML reports...
> > [2020-03-26T13:19:05.103Z] [htmlpublisher] Archiving at PROJECT level
> > /home/jenkins/jenkins-slave/workspace/Camel.website_PR-273/camel-
> > website/public to /x1/jenkins/jenkins-
> > home/jobs/Camel.website/branches/PR-273/htmlreports/Preview
> >
> > I think that means you can run the PR check on any executor, e.g.
> > 'ubuntu', just that the regular build needs to build on 'websites'.
> >
> > Note that this is independent of Vladimir's email discussing whether
> > PRs should be built on ASF resources or not, just a comment on how to
> > optimise executor usage.
> >
> > Thanks,
> > Robert
> >
> >
>
> --
> Gav...
--
Zoran Regvart
Re: More "websites" executors needed?
Posted by Gavin McDonald <ip...@gmail.com>.
Camels builds are definitely an issue and are consuming the 'websites'
label seemingly constantly from many PRs, minor changes
are triggering a new complete build each time. Builds are taking a minimum
of 20 minutes each time.
This needs to be looked into and see if there can be a better workflow.
Gav...
On Thu, Mar 26, 2020 at 4:40 PM Robert Munteanu <ro...@apache.org> wrote:
> Hello Zoran,
>
> On Thu, 2020-03-26 at 16:06 +0100, Zoran Regvart wrote:
> > Hi builders,
> > Apache Camel is participating in the Outreachy program[1], and we
> > have
> > a lot of pull requests[2] for which we run builds.
>
> That's cool and a nice problem to have :-)
>
> >
> > I think the other projects are having issues because of this. Not
> > sure
> > what can be done about this, we have no other way to check and
> > preview
> > the change if we don't run the build.
>
> I looked at the preview step of one of your builds and it seems that
> it's not actually publishing the changes anywhere, but instead
> archiving them in Jenkins
>
> [2020-03-26T13:19:05.101Z] [htmlpublisher] Archiving HTML reports...
> [2020-03-26T13:19:05.103Z] [htmlpublisher] Archiving at PROJECT level
> /home/jenkins/jenkins-slave/workspace/Camel.website_PR-273/camel-
> website/public to /x1/jenkins/jenkins-
> home/jobs/Camel.website/branches/PR-273/htmlreports/Preview
>
> I think that means you can run the PR check on any executor, e.g.
> 'ubuntu', just that the regular build needs to build on 'websites'.
>
> Note that this is independent of Vladimir's email discussing whether
> PRs should be built on ASF resources or not, just a comment on how to
> optimise executor usage.
>
> Thanks,
> Robert
>
>
--
Gav...
Re: More "websites" executors needed?
Posted by Robert Munteanu <ro...@apache.org>.
Hello Zoran,
On Thu, 2020-03-26 at 16:06 +0100, Zoran Regvart wrote:
> Hi builders,
> Apache Camel is participating in the Outreachy program[1], and we
> have
> a lot of pull requests[2] for which we run builds.
That's cool and a nice problem to have :-)
>
> I think the other projects are having issues because of this. Not
> sure
> what can be done about this, we have no other way to check and
> preview
> the change if we don't run the build.
I looked at the preview step of one of your builds and it seems that
it's not actually publishing the changes anywhere, but instead
archiving them in Jenkins
[2020-03-26T13:19:05.101Z] [htmlpublisher] Archiving HTML reports...
[2020-03-26T13:19:05.103Z] [htmlpublisher] Archiving at PROJECT level
/home/jenkins/jenkins-slave/workspace/Camel.website_PR-273/camel-
website/public to /x1/jenkins/jenkins-
home/jobs/Camel.website/branches/PR-273/htmlreports/Preview
I think that means you can run the PR check on any executor, e.g.
'ubuntu', just that the regular build needs to build on 'websites'.
Note that this is independent of Vladimir's email discussing whether
PRs should be built on ASF resources or not, just a comment on how to
optimise executor usage.
Thanks,
Robert
Re: More "websites" executors needed?
Posted by Zoran Regvart <zo...@regvart.com>.
Hi builders,
Apache Camel is participating in the Outreachy program[1], and we have
a lot of pull requests[2] for which we run builds.
I think the other projects are having issues because of this. Not sure
what can be done about this, we have no other way to check and preview
the change if we don't run the build.
zoran
[1] https://www.outreachy.org/apply/project-selection/#apache
[2] https://github.com/apache/camel-website/pulls
On Mon, Mar 23, 2020 at 4:42 PM Bertrand Delacretaz
<bd...@apache.org> wrote:
>
> Hi,
>
> Looking at https://builds.apache.org/label/websites/load-statistics
> IIUC there's only 2 executors for the "websites" label and they are
> very often both busy.
>
> We're regularly getting timeouts lately, waiting for such an executor
> in the Sling website build,
> https://builds.apache.org/job/Sling/job/sling-site/job/master/
>
> Do we need more executors, or is something else wrong?
>
> -Bertrand
--
Zoran Regvart