State of CI on builds.apache.org

[Bastian Krol <ba...@tu-dortmund.de>]

Hi folks,

as you might now, we run CouchDB builds on builds.apache.org for quite a 
while now.

Looking at https://builds.apache.org/job/CouchDB/ I see an ocean of red 
builds with a very occasional blue (successful) build in between. This 
makes me sad.

Actually, each build bubble represents six sub builds (3 operating 
systems, 2 Erlang versions). Anyway, obviously these builds are very 
unreliable and provide zero benefit to the project.

A breakdown of *why* the builds fail is here:

https://github.com/basti1302/couchdb-ci/blob/master/utils/analyze-jenkins-logs/ci-errors.markdown

(Caveat: This report is generated by a little script that makes some 
wild guesses and simplifications, so its results might be wrong)

My question: What can we do to make this better?

* Do we need to fix something in the various CouchDB source repos to 
make the build more reliable?
* Is the build itself the culprit? (e. g. Docker issues etc.)
* Will everything resolve itself if we switch back to one big repository?

The discussion "On dependency management and CI issues associated with 
it" brought up the point of dependency problems due to the multi-repo 
approach, but I really don't think that this is even the main issue on 
builds.a.o.

And, most importantly: Who is willing to help with this or better yet, 
take the lead here?

Best regards

Bastian

Re: State of CI on builds.apache.org

[Jan Lehnardt <ja...@apache.org>]

Bastian,

great initiative, thank you for the much needed kick in the behind.

Could we maybe get this hooked up to our Pull Request statuses on GitHub as well as notifications in IRC?

I think then this will get more attention :)

Best
Jan
--

> On 23 Apr 2016, at 08:00, Bastian Krol <ba...@tu-dortmund.de> wrote:
> 
> Hi folks,
> 
> as you might now, we run CouchDB builds on builds.apache.org for quite a while now.
> 
> Looking at https://builds.apache.org/job/CouchDB/ I see an ocean of red builds with a very occasional blue (successful) build in between. This makes me sad.
> 
> Actually, each build bubble represents six sub builds (3 operating systems, 2 Erlang versions). Anyway, obviously these builds are very unreliable and provide zero benefit to the project.
> 
> A breakdown of *why* the builds fail is here:
> 
> https://github.com/basti1302/couchdb-ci/blob/master/utils/analyze-jenkins-logs/ci-errors.markdown
> 
> (Caveat: This report is generated by a little script that makes some wild guesses and simplifications, so its results might be wrong)
> 
> My question: What can we do to make this better?
> 
> * Do we need to fix something in the various CouchDB source repos to make the build more reliable?
> * Is the build itself the culprit? (e. g. Docker issues etc.)
> * Will everything resolve itself if we switch back to one big repository?
> 
> The discussion "On dependency management and CI issues associated with it" brought up the point of dependency problems due to the multi-repo approach, but I really don't think that this is even the main issue on builds.a.o.
> 
> And, most importantly: Who is willing to help with this or better yet, take the lead here?
> 
> Best regards
> 
> Bastian

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/