[buildstream] 02/02: CI.rst: New file: Documentation for continuous integration

[tv...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tvb pushed a commit to branch jmac/performance-ci
in repository https://gitbox.apache.org/repos/asf/buildstream.git

commit 7b71f50312b53e90fd9f38a72a9a101a0e86e362
Author: Jim MacArthur <ji...@codethink.co.uk>
AuthorDate: Tue Mar 27 13:28:06 2018 +0100

    CI.rst: New file: Documentation for continuous integration
---
 CI.rst | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/CI.rst b/CI.rst
new file mode 100644
index 0000000..d32d152
--- /dev/null
+++ b/CI.rst
@@ -0,0 +1,30 @@
+Continuous integration and testing of BuildStream
+=================================================
+
+At the time of writing, BuildStream is hosted on GitLab's public website at https://gitlab.com/BuildStream/. It uses GitLab's continuous integration system to provide automated build and test of the main branch (`master`) and any branches pushed to the BuildStream repository.
+
+As a simplification, continuous testing consists of:
+
+* Creating a source distribution with `python3 setup.py sdist`
+* Extracting that distribution
+* Running `python3 setup.py test --addopts --integration`
+* Extracting the coverage results and marking them as a test artifact.
+
+These tests are run on in both Linux and Unix mode (Unix is not actually used to test Unix mode, but BST_FORCE_BACKEND: "unix" is used to test as much of that mode as we can).
+
+Performance tests are also run in Linux mode. Performance tests operate in a different manner, do not use the source distribution, and require a special runner to keep performance tests consistent. Performance tests are performed using the separate `benchmarks` repository from https://gitlab.com/BuildStream/benchmarks and the performance test procedure is defined by that repository, not by BuildStream.
+
+`.gitlab-ci.yml` contains the full details of the test procedure.
+
+
+Requirements
+============
+The project must be hosted on a GitLab instance for the `.gitlab-ci.yml` script to work.
+
+The public GitLab instance (https://gitlab.com/) has shared runners which are suitable for the functional tests. Shared runners must be enabled for the BuildStream project for this to work.
+
+To run the performance tests, one or more reference hardware systems must act as a runner for the BuildStream project. These runners should only process jobs with the tag 'benchmarks'. They should be set up to run one job at a time. They should be set to use the `shell` Executor.
+
+Instructions on setting up GitLab runners are provided by GitLab at https://docs.gitlab.com/runner/install/.
+
+Assuming BuildStream and the `benchmarks` repository are both hosted on GitLab, the performance runner machines should be enabled for *both* projects. This will enable you to run continuous testing for BuildStream and also test the benchmarking repository itself and compare results.