You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ctest (Jira)" <ji...@apache.org> on 2020/04/16 04:46:00 UTC
[jira] [Updated] (HADOOP-16971) testFileContextResolveAfs creates
dangling link and fails for subsequent runs
[ https://issues.apache.org/jira/browse/HADOOP-16971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ctest updated HADOOP-16971:
---------------------------
Component/s: fs
Description:
In the test testFileContextResolveAfs, the symlink TestFileContextResolveAfs2 (linked to TestFileContextResolveAfs1) cannot be deleted when the test finishes.
This is because TestFileContextResolveAfs1 was always deleted before TestFileContextResolveAfs2 when they were both passed into FileSystem#deleteOnExit. This caused TestFileContextResolveAfs2 to become a dangling link, which FileSystem in Hadoop currently cannot delete. (This is because Files#exists will return false for dangling links.)
As a result, the test `testFileContextResolveAfs` only passed for the first run. And for later runs of this test, it will fail by throwing the following exception:
{code:java}
java.io.IOException: Error 1 creating symlink file:/*/hadoop-common-project/hadoop-common/target/test/data/TestFileContextResolveAfs2 to /*/hadoop-common-project/hadoop-common/target/test/data/TestFileContextResolveAfs1
{code}
was:
In the test `testFileContextResolveAfs`, the symlink `TestFileContextResolveAfs2` (linked to `TestFileContextResolveAfs1`) was not deleted as intended in the first run, thus the test will fail in the second run.
The reason is that this test uses org.apache.hadoop.fs.FileSystem to handle the deletion of symlink, which
1. does not support symlink.
2. deletes `TestFileContextResolveAfs1` before `TestFileContextResolveAfs2` if both links passed into `deleteOnExit`. This is because of in paths in the set `deleteOnExit`, `TestFileContextResolveAfs1` will always be deleted before `TestFileContextResolveAfs2`, due to TreeSet property.
When `TestFileContextResolveAfs1` has been deleted, `TestFileContextResolveAfs2` became an orphan symlink and is considered as a non-exisitent path by org.apache.hadoop.fs.FileSystem#exists, thus its deletion cannot be completed.
When the test is run for the second time, it will fail with IOException because `TestFileContextResolveAfs2` has been created.
Summary: testFileContextResolveAfs creates dangling link and fails for subsequent runs (was: testFileContextResolveAfs failed to delete created symlink and pollute subsequent test run.)
> testFileContextResolveAfs creates dangling link and fails for subsequent runs
> -----------------------------------------------------------------------------
>
> Key: HADOOP-16971
> URL: https://issues.apache.org/jira/browse/HADOOP-16971
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, fs, test
> Affects Versions: 3.2.1, 3.4.0
> Reporter: Ctest
> Priority: Minor
> Labels: easyfix
> Attachments: HADOOP-16971.000.patch
>
>
> In the test testFileContextResolveAfs, the symlink TestFileContextResolveAfs2 (linked to TestFileContextResolveAfs1) cannot be deleted when the test finishes.
> This is because TestFileContextResolveAfs1 was always deleted before TestFileContextResolveAfs2 when they were both passed into FileSystem#deleteOnExit. This caused TestFileContextResolveAfs2 to become a dangling link, which FileSystem in Hadoop currently cannot delete. (This is because Files#exists will return false for dangling links.)
> As a result, the test `testFileContextResolveAfs` only passed for the first run. And for later runs of this test, it will fail by throwing the following exception:
> {code:java}
> java.io.IOException: Error 1 creating symlink file:/*/hadoop-common-project/hadoop-common/target/test/data/TestFileContextResolveAfs2 to /*/hadoop-common-project/hadoop-common/target/test/data/TestFileContextResolveAfs1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org