You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2023/08/21 19:39:00 UTC

[jira] [Created] (IMPALA-12389) Use -skipTrash for hdfs_util.py's HadoopFsCommandLineClient::delete_file_dir

Joe McDonnell created IMPALA-12389:
--------------------------------------

             Summary: Use -skipTrash for hdfs_util.py's HadoopFsCommandLineClient::delete_file_dir
                 Key: IMPALA-12389
                 URL: https://issues.apache.org/jira/browse/IMPALA-12389
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
    Affects Versions: Impala 4.3.0
            Reporter: Joe McDonnell


When tests use unique databases, there is some logic to clean up the database directory when the test is done. This logic uses delete_file_dir. On some filesystem types like S3, this is implemented with HadoopFsCommandLineClient.
{noformat}
  def delete_file_dir(self, path, recursive=False):
    """Delete the file or directory given by the specified path. Recursive must be true
    for directories."""
    rm_command = ['-rm', path]
    if recursive:
      rm_command = ['-rm', '-r', path]
    (status, stdout, stderr) = self._hadoop_fs_shell(rm_command)
    return status == 0{noformat}
We should change this to add "-skipTrash" to avoid accumulating files in the trash directory. This is particularly important for S3, because the trash directory is currently not cleaned up by bin/jenkins/release_cloud_resources.sh.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org