[buildstream] 02/02: downloadablefilesource.py: Fix race condition creating mirror directory

[tv...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tvb pushed a commit to branch tristan/fix-downloadablefile-race
in repository https://gitbox.apache.org/repos/asf/buildstream.git

commit d0cfb0e25949e03110013a5c93e5afd6737c700c
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Sat Mar 4 16:14:04 2023 +0900

    downloadablefilesource.py: Fix race condition creating mirror directory
    
    This file has for some time been dangerously creating it's mirror directory
    only if it doesn't yet exist, presenting a race condition for concurrent
    sources tracking the same file (at possibly different versions).
    
    Fixes #1831
---
 src/buildstream/downloadablefilesource.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/src/buildstream/downloadablefilesource.py b/src/buildstream/downloadablefilesource.py
index e31033506..5f44a2ff1 100644
--- a/src/buildstream/downloadablefilesource.py
+++ b/src/buildstream/downloadablefilesource.py
@@ -235,8 +235,12 @@ class DownloadableFileSource(Source):
                 return self.ref
 
             # Make sure url-specific mirror dir exists.
-            if not os.path.isdir(self._mirror_dir):
-                os.makedirs(self._mirror_dir)
+            try:
+                os.makedirs(self._mirror_dir, exist_ok=True)
+            except FileExistsError as e:
+                raise SourceError(
+                    "{}: Mirror directory exists but is not a directory: {}".format(self, self._mirror_dir)
+                ) from e
 
             # Store by sha256sum
             sha256 = utils.sha256sum(local_file)