You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by ad...@apache.org on 2021/09/01 14:36:12 UTC
[cassandra] branch cassandra-3.0 updated: Add resource flags to
CircleCI config generation script
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new e4b37c3 Add resource flags to CircleCI config generation script
e4b37c3 is described below
commit e4b37c3271c0e91407816d3c5370ebc8b95a615a
Author: Andrés de la Peña <a....@gmail.com>
AuthorDate: Fri Aug 20 11:34:01 2021 +0100
Add resource flags to CircleCI config generation script
patch by Andrés de la Peña; reviewed by Ekaterina Dimitrova for CASSANDRA-16871
---
.circleci/generate.sh | 95 ++++++++++++++++++++++++++++++++++++++++-----------
.circleci/readme.md | 46 ++++++++++++++++++-------
2 files changed, 110 insertions(+), 31 deletions(-)
diff --git a/.circleci/generate.sh b/.circleci/generate.sh
index 832c63f..db7d772 100755
--- a/.circleci/generate.sh
+++ b/.circleci/generate.sh
@@ -19,23 +19,80 @@
BASEDIR=`dirname $0`
-# setup lowres
-circleci config process $BASEDIR/config-2_1.yml > $BASEDIR/config.yml.LOWRES.tmp
-cat $BASEDIR/license.yml $BASEDIR/config.yml.LOWRES.tmp > $BASEDIR/config.yml.LOWRES
-rm $BASEDIR/config.yml.LOWRES.tmp
-
-# setup midres
-patch -o $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.mid_res.patch
-circleci config process $BASEDIR/config-2_1.yml.MIDRES > $BASEDIR/config.yml.MIDRES.tmp
-cat $BASEDIR/license.yml $BASEDIR/config.yml.MIDRES.tmp > $BASEDIR/config.yml.MIDRES
-rm $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config.yml.MIDRES.tmp
-
-# setup highres
-patch -o $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.high_res.patch
-circleci config process $BASEDIR/config-2_1.yml.HIGHRES > $BASEDIR/config.yml.HIGHRES.tmp
-cat $BASEDIR/license.yml $BASEDIR/config.yml.HIGHRES.tmp > $BASEDIR/config.yml.HIGHRES
-rm $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config.yml.HIGHRES.tmp
-
-# copy lower into config.yml to make sure this gets updated
-cp $BASEDIR/config.yml.LOWRES $BASEDIR/config.yml
+die ()
+{
+ echo "ERROR: $*"
+ echo "Usage: $0 [-l|-m|-h]"
+ echo " -l Generate config.yml using low resources"
+ echo " -m Generate config.yml using mid resources"
+ echo " -h Generate config.yml using high resources"
+ echo " No flags generates the default config.yml using low resources and the three"
+ echo " templates (config.yml.LOWRES, config.yml.MIDRES and config.yml.HIGHRES)"
+ exit 1
+}
+
+lowres=false
+midres=false
+highres=false
+while getopts ":lmh" opt; do
+ case $opt in
+ l ) ($midres || $highres) && die "Cannot specify option -l after specifying options -m or -h"
+ lowres=true
+ ;;
+ m ) ($lowres || $highres) && die "Cannot specify option -m after specifying options -l or -h"
+ midres=true
+ ;;
+ h ) ($lowres || $midres) && die "Cannot specify option -h after specifying options -l or -m"
+ highres=true
+ ;;
+ \?) die "Invalid option: -$OPTARG"
+ ;;
+ esac
+done
+
+if $lowres; then
+ echo "Generating new config.yml file with low resources from config-2_1.yml"
+ circleci config process $BASEDIR/config-2_1.yml > $BASEDIR/config.yml.LOWRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.LOWRES.tmp > $BASEDIR/config.yml
+ rm $BASEDIR/config.yml.LOWRES.tmp
+
+elif $midres; then
+ echo "Generating new config.yml file with middle resources from config-2_1.yml"
+ patch -o $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.mid_res.patch
+ circleci config process $BASEDIR/config-2_1.yml.MIDRES > $BASEDIR/config.yml.MIDRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.MIDRES.tmp > $BASEDIR/config.yml
+ rm $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config.yml.MIDRES.tmp
+
+elif $highres; then
+ echo "Generating new config.yml file with high resources from config-2_1.yml"
+ patch -o $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.high_res.patch
+ circleci config process $BASEDIR/config-2_1.yml.HIGHRES > $BASEDIR/config.yml.HIGHRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.HIGHRES.tmp > $BASEDIR/config.yml
+ rm $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config.yml.HIGHRES.tmp
+
+else
+ echo "Generating new config.yml file with low resources and LOWRES/MIDRES/HIGHRES templates from config-2_1.yml"
+
+ # setup lowres
+ circleci config process $BASEDIR/config-2_1.yml > $BASEDIR/config.yml.LOWRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.LOWRES.tmp > $BASEDIR/config.yml.LOWRES
+ rm $BASEDIR/config.yml.LOWRES.tmp
+
+ # setup midres
+ patch -o $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.mid_res.patch
+ circleci config process $BASEDIR/config-2_1.yml.MIDRES > $BASEDIR/config.yml.MIDRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.MIDRES.tmp > $BASEDIR/config.yml.MIDRES
+ rm $BASEDIR/config-2_1.yml.MIDRES $BASEDIR/config.yml.MIDRES.tmp
+
+ # setup highres
+ patch -o $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config-2_1.yml $BASEDIR/config-2_1.yml.high_res.patch
+ circleci config process $BASEDIR/config-2_1.yml.HIGHRES > $BASEDIR/config.yml.HIGHRES.tmp
+ cat $BASEDIR/license.yml $BASEDIR/config.yml.HIGHRES.tmp > $BASEDIR/config.yml.HIGHRES
+ rm $BASEDIR/config-2_1.yml.HIGHRES $BASEDIR/config.yml.HIGHRES.tmp
+
+ # copy lower into config.yml to make sure this gets updated
+ cp $BASEDIR/config.yml.LOWRES $BASEDIR/config.yml
+fi
+
+
diff --git a/.circleci/readme.md b/.circleci/readme.md
index 3c4d963..e4e32f5 100644
--- a/.circleci/readme.md
+++ b/.circleci/readme.md
@@ -20,24 +20,46 @@
# CircleCI config files
-## Switching to high resource settings
-This directory contains generated files for high and low resource settings. Switch
-between them by copying the correct file to config.yml and committing the result;
+This directory contains the configuration for CircleCI continous integration platform.
+The file `config.yml` is the configuration file that is read by CircleCI. This file is
+automatically generated by the `generate.sh` script from the `config-2_1.yml` file.
+
+The provided `config.yml` file uses low resources so users of the CircleCI free tier can
+use it. Additionally, there are three versions of this file using different resources so
+users who have access to premium CircleCI resources can use larger instances and more
+parallelism. These files are `config.yml.LOWRES`, `config.yml.MIDRES` and `config.yml.HIGHRES`.
+The default `config.yml` file is just a copy of `config.yml.LOWRES`.
+
+## Switching to higher resource settings
+This directory contains generated files for low, middle and high resource settings.
+Switch between them by copying the correct file to `config.yml` and committing the result:
`cp .circleci/config.yml.HIGHRES .circleci/config.yml`
-config.yml.LOWRES is the default config. MIDRES and HIGHRES are custom configs for those who
-have access to premium CircleCI resources.
+Alternatively, you can run the `generate.sh` script with the flags `-l`/`-m`/`-h`
+to regenerate the `config.yml` file from `config-2_1.yml` using LOWRES/MIDRES/HIGHRES.
+This script validates and applies any changes to the `config-2_1.yml` file, and it
+requires the [CircleCI CLI](https://circleci.com/docs/2.0/local-cli/#install) to be
+installed.
-Make sure you never edit the config.yml manually.
+## Updating the config
+For configuration changes meant to be permanent in the Apache repo you should never edit
+the `config.yml` file manually. Instead, you should edit the `config-2_1.yml` file and then
+regenerate the `config.yml`, `config.yml.LOWRES`, `config.yml.MIDRES` and `config.yml.HIGHRES`
+files by runnining the `generate.sh` script without any flags. For using this script you
+need to install the [CircleCI CLI](https://circleci.com/docs/2.0/local-cli/#install).
+
+As for temporal changes done while working in a patch, such as pointing to you dtest repo or
+running a test repeatedly, you can either directly edit `config.yml` or edit `config-2_1.yml`
+and then regenerate `config.yml` with the `generate.sh` script using a `-l`/`-m`/`-h` flag.
+When this flag is used only the `config.yml` will be generated.
-## Updating the config master
-To update the config (other than just swapping high/mid/low resources) you need to install
-the [CircleCI CLI](https://circleci.com/docs/2.0/local-cli/#install).
+Please note that any previous swapping or edition of the generated files will be overriden
+by running `generate.sh` without arguments, returning `config.yml` to the default LOWRES. So if
+you previously swapped your `config.yml` to MIDRES or HIGHRES you would need to either swap it
+again or use the `-l`/`-m`/`-h` script flags.
-The directory contains `config-2_1.yml` which is then converted to the actual HIGH/MID/LOW
-resource files. There is a script called `generate.sh` which creates the LOWRES, MIDRES, and
-HIGHRES files, read below for details how to do it manually;
+Read below for details how to generate the files manually without the `generate.sh` script:
1. make your edits to config-2_1.yml - let it stay at lowres settings
1. generate a valid LOWRES file:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org