[jira] [Created] (IMPALA-12720) Investigate reference deleted hdfs files in spilling tests

["Yida Wu (Jira)" <ji...@apache.org>]

Yida Wu created IMPALA-12720:
--------------------------------

             Summary: Investigate reference deleted hdfs files in spilling tests
                 Key: IMPALA-12720
                 URL: https://issues.apache.org/jira/browse/IMPALA-12720
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 4.3.0
            Reporter: Yida Wu
            Assignee: Yida Wu


This is following jira for IMPALA-12698. While running the test TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the immediate release of file handles. However, in local tests, it was noticed that the files are eventually deleted from the fd path after a few minutes. Further investigation may be worthwhile to understand the underlying cause of this symptom.
{code:java}
lr-x------ 1 impala impala 64 Jan  9 23:40 750 -> '/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta (deleted)'
lr-x------ 1 impala impala 64 Jan  9 23:40 751 -> '/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938 (deleted)'
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org