You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by ra...@apache.org on 2020/11/06 17:02:20 UTC
[trafficcontrol] branch master updated: GitHub Actions: Cache CPAN
modules (#5253)
This is an automated email from the ASF dual-hosted git repository.
rawlin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/master by this push:
new bfcf04a GitHub Actions: Cache CPAN modules (#5253)
bfcf04a is described below
commit bfcf04a3f1b99ec72d553a9b94585282b3bf83dd
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Fri Nov 6 10:02:08 2020 -0700
GitHub Actions: Cache CPAN modules (#5253)
* Copy local Traffic Ops Perl library into Traffic Ops image
* Cache Traffic Ops Perl CPAN modules
* Do not install Carton locally
* Run systemwide `carton` the second time, too
---
.github/actions/run-ciab/run-ciab.sh | 2 ++
.github/workflows/ciab.yaml | 7 +++++++
.gitignore | 3 +++
docs/source/admin/quick_howto/ciab.rst | 7 +++++++
infrastructure/cdn-in-a-box/traffic_ops/Dockerfile | 9 ++++++---
infrastructure/cdn-in-a-box/traffic_ops/Dockerfile.dockerignore | 1 +
traffic_ops/app/local/.gitkeep | 0
7 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/.github/actions/run-ciab/run-ciab.sh b/.github/actions/run-ciab/run-ciab.sh
index 64060ed..eedb43d 100755
--- a/.github/actions/run-ciab/run-ciab.sh
+++ b/.github/actions/run-ciab/run-ciab.sh
@@ -32,6 +32,8 @@ other_services='dns edge enroller mid-01 mid-02 origin trafficmonitor trafficops
docker_compose='docker-compose -f ./docker-compose.yml -f ./docker-compose.readiness.yml';
$docker_compose up -d $logged_services $other_services;
$docker_compose logs -f $logged_services &
+# Copy built Perl modules for caching
+docker cp "$(docker-compose ps -q trafficops-perl):/opt/traffic_ops/app/local" "${GITHUB_WORKSPACE}/traffic_ops/app" &
echo 'Waiting for the readiness container to exit...';
if ! timeout 12m $docker_compose logs -f readiness >/dev/null; then
diff --git a/.github/workflows/ciab.yaml b/.github/workflows/ciab.yaml
index 5703b8c..faa0532 100644
--- a/.github/workflows/ciab.yaml
+++ b/.github/workflows/ciab.yaml
@@ -210,6 +210,13 @@ jobs:
uses: actions/download-artifact@v2
with:
path: ${{ github.workspace }}/dist/
+ - name: Cache Perl modules
+ uses: actions/cache@v2
+ with:
+ path: ${{ github.workspace }}/traffic_ops/app/local
+ key: ${{ runner.os }}-cpan-${{ hashFiles('cache/**.tar.gz') }}
+ restore-keys: |
+ ${{ runner.os }}-cpan-
- name: Build CDN-in-a-Box images
uses: ./.github/actions/build-ciab
- name: Start CDN-in-a-Box
diff --git a/.gitignore b/.gitignore
index d05407f..ca5ec4a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,9 @@ docs/build/*
/grove/grove
/grove/grovetccfg/grovetccfg
local
+!/traffic_ops/app/local/
+/traffic_ops/app/local/*
+!/traffic_ops/app/local/.gitkeep
traffic_ops_extensions/log/*
traffic_ops_extensions/lib/Extensions/InfluxDB/log/*
traffic_ops/app/log/*.log
diff --git a/docs/source/admin/quick_howto/ciab.rst b/docs/source/admin/quick_howto/ciab.rst
index 805ef8c..bb1d0f5 100644
--- a/docs/source/admin/quick_howto/ciab.rst
+++ b/docs/source/admin/quick_howto/ciab.rst
@@ -44,6 +44,13 @@ These can all be supplied manually via the steps in :ref:`dev-building` (for Tra
.. tip:: When updating CDN-in-a-Box, there is no need to remove old images before building new ones. Docker detects which files are updated and only reuses cached layers that have not changed.
+The image that takes the takes the longest to build is the ``trafficops-perl`` image. In order to avoid needing to download, build, and test 239 Perl CPAN modules each time you rebuild the image from scratch, you can run the following command while running CDN in a Box in order to skip building the Perl modules next time:
+
+.. code-block:: shell
+ :caption: Make a local copy of CPAN modules used by Traffic Ops Perl
+
+ docker cp $(docker-compose ps -q trafficops-perl):/opt/traffic_ops/app/local ../../traffic_ops/app
+
Usage
-----
In a typical scenario, if the steps in `Building`_ have been followed, all that's required to start the CDN in a Box is to run ``docker-compose up`` - optionally with the ``-d`` flag to run without binding to the terminal - from the :file:`infrastructure/cdn-in-a-box/` directory. This will start up the entire stack and should take care of any needed initial configuration. The services within the environment are by default not exposed locally to the host. If this is the desired behavior w [...]
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile b/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile
index 6485182..2199bbb 100644
--- a/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile
+++ b/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile
@@ -79,8 +79,11 @@ ADD traffic_router/core/src/test/resources/geo/GeoLite2-City.mmdb.gz /opt/traffi
WORKDIR /opt/traffic_ops/app
ADD traffic_ops/app/cpanfile traffic_ops/install/bin/install_goose.sh ./
-RUN cpanm -l ./local Carton && \
- POSTGRES_HOME=/usr/pgsql-9.6 PERL5LIB=$(pwd)/local/lib/perl5 ./local/bin/carton && \
+
+# Start with the existing traffic_ops/app/local directory
+COPY traffic_ops/app/local local
+RUN cpanm Carton && \
+ POSTGRES_HOME=/usr/pgsql-9.6 PERL5LIB=$(pwd)/local/lib/perl5 carton && \
rm -rf $HOME/.cpan* /tmp/Dockerfile /tmp/local.tar.gz ./cpanfile
RUN ./install_goose.sh
@@ -94,7 +97,7 @@ RUN rpm -Uvh /$(basename $TRAFFIC_OPS_RPM) && \
# Run carton again, in case the cpanfile included in the RPM differs from the one used earlier in the
# build (should never happen - Perl is supposed to be going away)
-RUN POSTGRES_HOME=/usr/pgsql-9.6 PERL5LIB=$(pwd)/local/lib/perl5 ./local/bin/carton && \
+RUN POSTGRES_HOME=/usr/pgsql-9.6 PERL5LIB=$(pwd)/local/lib/perl5 carton && \
rm -rf $HOME/.cpan* /tmp/Dockerfile /tmp/local.tar.gz
ADD infrastructure/cdn-in-a-box/enroller/server_template.json \
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile.dockerignore b/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile.dockerignore
index c6cb663..5e6a5a2 100644
--- a/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile.dockerignore
+++ b/infrastructure/cdn-in-a-box/traffic_ops/Dockerfile.dockerignore
@@ -25,5 +25,6 @@
*
!infrastructure/cdn-in-a-box/
!traffic_ops/app/cpanfile
+!traffic_ops/app/local/
!traffic_ops/install/bin/install_goose.sh
!traffic_router/core/src/test/resources/geo/GeoLite2-City.mmdb.gz
diff --git a/traffic_ops/app/local/.gitkeep b/traffic_ops/app/local/.gitkeep
new file mode 100644
index 0000000..e69de29