You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2023/08/21 19:39:00 UTC
[jira] [Created] (IMPALA-12389) Use -skipTrash for hdfs_util.py's HadoopFsCommandLineClient::delete_file_dir
Joe McDonnell created IMPALA-12389:
--------------------------------------
Summary: Use -skipTrash for hdfs_util.py's HadoopFsCommandLineClient::delete_file_dir
Key: IMPALA-12389
URL: https://issues.apache.org/jira/browse/IMPALA-12389
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 4.3.0
Reporter: Joe McDonnell
When tests use unique databases, there is some logic to clean up the database directory when the test is done. This logic uses delete_file_dir. On some filesystem types like S3, this is implemented with HadoopFsCommandLineClient.
{noformat}
def delete_file_dir(self, path, recursive=False):
"""Delete the file or directory given by the specified path. Recursive must be true
for directories."""
rm_command = ['-rm', path]
if recursive:
rm_command = ['-rm', '-r', path]
(status, stdout, stderr) = self._hadoop_fs_shell(rm_command)
return status == 0{noformat}
We should change this to add "-skipTrash" to avoid accumulating files in the trash directory. This is particularly important for S3, because the trash directory is currently not cleaned up by bin/jenkins/release_cloud_resources.sh.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org