[GitHub] [buildstream] juergbi commented on a diff in pull request #1793: Use copy_file_range when exists

[GitBox <gi...@apache.org>]

juergbi commented on code in PR #1793:
URL: https://github.com/apache/buildstream/pull/1793#discussion_r1038781304


##########
src/buildstream/utils.py:
##########
@@ -359,6 +359,18 @@ def sha256sum(filename: str) -> str:
     return h.hexdigest()
 
 
+try:
+    os.copy_file_range  # type: ignore[attr-defined] Requires Python 3.8 or newer
+except AttributeError:
+    _copy_file = lambda src, dest: shutil.copyfile(src, dest)
+else:
+    def _copy_file(src, dest):
+        with open(src, "rb") as src_file, open(dest, "wb") as dest_file:
+            num_bytes = os.fstat(src_file.fileno()).st_size
+            while num_bytes > 0:
+                num_bytes -= os.copy_file_range(src_file.fileno(), dest_file.fileno(), num_bytes)

Review Comment:
   `copy_file_range()` may fail if the kernel is too old. The syscall was introduced in Linux 4.5 and support for cross-filesystem copies was added in Linux 5.3. Current glibc does not provide a user-space emulation if the kernel doesn't support it.
   
   I.e. at least if `copy_file_range()` fails with `ENOSYS` or `EXDEV`, we need to support fallback to `shutil.copyfile()`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org