[impala] branch master updated: IMPALA-11471: Track disk usage for build-all-flag-combinations.sh

[st...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new 5773b8e95 IMPALA-11471: Track disk usage for build-all-flag-combinations.sh
5773b8e95 is described below

commit 5773b8e95683a4ca04e4dfe92394392b53cb3256
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Tue Aug 2 20:54:00 2022 -0700

    IMPALA-11471: Track disk usage for build-all-flag-combinations.sh
    
    This adds some calls to df and du to track disk space
    usage throughout the builds. This also cleans up the
    Impala dev environment before creating the m2 archive.
    
    Change-Id: I8ab31d8d7096b49d8404edf7521d46f23155526f
    Reviewed-on: http://gerrit.cloudera.org:8080/18810
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Joe McDonnell <jo...@cloudera.com>
---
 bin/jenkins/build-all-flag-combinations.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/bin/jenkins/build-all-flag-combinations.sh b/bin/jenkins/build-all-flag-combinations.sh
index 94acd94fc..8dc76f93b 100755
--- a/bin/jenkins/build-all-flag-combinations.sh
+++ b/bin/jenkins/build-all-flag-combinations.sh
@@ -90,6 +90,10 @@ for CONFIG in "${CONFIGS[@]}"; do
   fi
   ccache -s
   bin/jenkins/get_maven_statistics.sh logs/mvn/mvn.log
+  # Dump the current disk usage and check the usage for
+  # several heavy users of disk space.
+  df
+  du -s ~/.m2 ./be ./toolchain
 
   # Keep each maven log from each round of the build
   cp logs/mvn/mvn.log "${TMP_DIR}/mvn.$(date +%s.%N).log"
@@ -131,6 +135,10 @@ else
 
   ccache -s
   bin/jenkins/get_maven_statistics.sh logs/mvn/mvn.log
+  # Dump the current disk usage and check the usage for
+  # several heavy users of disk space.
+  df
+  du -s ~/.m2 ./be ./toolchain
 
   # Keep each maven log from each round of the build
   cp logs/mvn/mvn.log "${TMP_DIR}/mvn.$(date +%s.%N).log"
@@ -138,6 +146,10 @@ else
   cat logs/mvn/mvn.log >> "${TMP_DIR}/mvn_accumulated.log"
 fi
 
+# To avoid diskspace issues, clean up the Impala build directory
+# before creating the archive.
+rm -rf be/build
+
 # Restore the maven logs (these don't interfere with existing mvn.log)
 # These files may not exist if this is doing a dryrun.
 if ls ${TMP_DIR}/mvn* > /dev/null 2>&1 ; then
@@ -152,6 +164,11 @@ then
 fi
 
 if [[ "$GENERATE_M2_ARCHIVE" == true ]]; then
+  echo "Generating m2 archive."
+
+  # Print current diskspace
+  df
+
   # Make a tarball of the .m2 directory
   bin/jenkins/archive_m2_directory.sh logs/mvn/mvn_accumulated.log logs/m2_archive.tar.gz
 fi