[airflow] branch main updated: Improve isolation of MSSQL files (#18538)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 932c3b5  Improve isolation of MSSQL files (#18538)
932c3b5 is described below

commit 932c3b5df5444970484f0ec23589be1820a3270d
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Sep 27 10:01:37 2021 +0200

    Improve isolation of MSSQL files (#18538)
    
    The MSSQL tests sometimes failed with "unhealthy" status. The reaso for
    that was that our cleanup did not properly clean hidden files I think.
    We were potentially reusing the bind volumes for MSSQL tests, but it
    turned out that we did not remove hidden files from that volume and that
    likely caused problems accross 2017/2019 versions when the volumes were
    reused.
    
    This change implements two-fold protection against such case:
    * the volumes have different path for different MSSQL versions
    * hidden files are also removed
---
 scripts/ci/testing/ci_run_single_airflow_test_in_docker.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/ci/testing/ci_run_single_airflow_test_in_docker.sh b/scripts/ci/testing/ci_run_single_airflow_test_in_docker.sh
index bc4d296..f8f6d98 100755
--- a/scripts/ci/testing/ci_run_single_airflow_test_in_docker.sh
+++ b/scripts/ci/testing/ci_run_single_airflow_test_in_docker.sh
@@ -93,7 +93,7 @@ function run_airflow_testing_in_docker() {
             # so we need to mount an external volume for its db location
             # the external db must allow for parallel testing so TEST_TYPE
             # is added to the volume name
-            export MSSQL_DATA_VOLUME="${HOME}/tmp-mssql-volume-${TEST_TYPE}"
+            export MSSQL_DATA_VOLUME="${HOME}/tmp-mssql-volume-${TEST_TYPE}-${MSSQL_VERSION}"
             mkdir -p "${MSSQL_DATA_VOLUME}"
             # MSSQL 2019 runs with non-root user by default so we have to make the volumes world-writeable
             # This is a bit scary and we could get by making it group-writeable but the group would have
@@ -102,10 +102,10 @@ function run_airflow_testing_in_docker() {
             backend_docker_compose+=("-f" "${SCRIPTS_CI_DIR}/docker-compose/backend-mssql-bind-volume.yml")
 
             # Runner user doesn't have blanket sudo access, but we can run docker as root. Go figure
-            traps::add_trap "docker run -u 0 --rm -v ${MSSQL_DATA_VOLUME}:/mssql alpine sh -c 'rm -rvf -- /mssql/*' || true" EXIT
+            traps::add_trap "docker run -u 0 --rm -v ${MSSQL_DATA_VOLUME}:/mssql alpine sh -c 'rm -rvf -- /mssql/.* /mssql/*' || true" EXIT
 
-            # Clean up at start too, in case a previous runer left it messy
-            docker run --rm -u 0 -v "${MSSQL_DATA_VOLUME}":/mssql alpine sh -c 'rm -rfv -- /mssql/*'  || true
+            # Clean up at start too, in case a previous runner left it messy
+            docker run --rm -u 0 -v "${MSSQL_DATA_VOLUME}":/mssql alpine sh -c 'rm -rfv -- /mssql/.* /mssql/*'  || true
         else
             backend_docker_compose+=("-f" "${SCRIPTS_CI_DIR}/docker-compose/backend-mssql-docker-volume.yml")
         fi