[general] Establishing build, test and release process

[Stepan Mishura <st...@gmail.com>]

Hi all,

The process is aimed to start publishing regularly builds that are
stable and well tested.
Currently we build and publish Harmony snapshots quite regularly. And
I think it is a time to organize the same for developer's builds.

I think that everybody agree that the main criteria for promoting a
build as developer's build its testing status. There are two kinds of
testing: automated and manual. I think we can easily organize
automated testing for snapshots that is based on the build-and-test
infra.

I've updated the infra a bit to add a CC project that downloads and
unpacks a snapshot. Once the snapshot is downloaded CC triggers all
testing scenarios to be run against the downloaded snapshot. I've
setup CC on Linux_x86 a Windows_x86 for the following tests:
classlib-test, dacapo, scimark, jetty scenario and tomcat scenario.
And run them on snapshot r540000. The testing result are available
here[1].

So I'd like to propose the following process: we setup CC infra on
different platforms for testing published snapshots. For each snapshot
there should be a page on the site which contains all testing results
(I think it is enough to keep results for 4-5 latest snapshots). Also
the page may contain a link to a separate wiki page. The wiki page
should contain testing results that were run manually by everybody.
This should give a picture about a selected snapshot:
stable/unstable/broken/has minor issues and so on . If we find that a
snapshot is good enough we may promote it as developer's build.

Thoughts?

[1] http://people.apache.org/~smishura/r540000/index.html

Thanks,
Stepan Mishura
Intel Enterprise Solutions Software Division

Re: [general] Establishing build, test and release process

[Mikhail Loenko <ml...@gmail.com>]

2007/5/23, Stepan Mishura <st...@gmail.com>:
> Hi all,
>
> The process is aimed to start publishing regularly builds that are
> stable and well tested.
> Currently we build and publish Harmony snapshots quite regularly. And
> I think it is a time to organize the same for developer's builds.
>
> I think that everybody agree that the main criteria for promoting a
> build as developer's build its testing status. There are two kinds of
> testing: automated and manual. I think we can easily organize
> automated testing for snapshots that is based on the build-and-test
> infra.
>
> I've updated the infra a bit to add a CC project that downloads and
> unpacks a snapshot. Once the snapshot is downloaded CC triggers all
> testing scenarios to be run against the downloaded snapshot. I've
> setup CC on Linux_x86 a Windows_x86 for the following tests:
> classlib-test, dacapo, scimark, jetty scenario and tomcat scenario.
> And run them on snapshot r540000. The testing result are available
> here[1].
>
> So I'd like to propose the following process: we setup CC infra on
> different platforms for testing published snapshots. For each snapshot
> there should be a page on the site which contains all testing results
> (I think it is enough to keep results for 4-5 latest snapshots). Also
> the page may contain a link to a separate wiki page. The wiki page
> should contain testing results that were run manually by everybody.
> This should give a picture about a selected snapshot:
> stable/unstable/broken/has minor issues and so on . If we find that a
> snapshot is good enough we may promote it as developer's build.
>
> Thoughts?

That's cool, thanks for doing this!

it might make sense to create a single page containing all the results,
or we can add a frame to for example web site that will contain wiki page.

Thanks,
Mikhail

>
> [1] http://people.apache.org/~smishura/r540000/index.html
>
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division
>