You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jimmy Zuber (Jira)" <ji...@apache.org> on 2020/06/30 17:41:00 UTC
[jira] [Created] (HADOOP-17105) S3AFS globStatus attempts to
resolve symlinks
Jimmy Zuber created HADOOP-17105:
------------------------------------
Summary: S3AFS globStatus attempts to resolve symlinks
Key: HADOOP-17105
URL: https://issues.apache.org/jira/browse/HADOOP-17105
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Jimmy Zuber
The S3AFileSystem implementation of the globStatus API has a setting configured to resolve symlinks. Under certain circumstances, this will cause additional file existence checks to be performed in order to determine if a FileStatus signifies a symlink. As symlinks are not supported in S3AFileSystem, these calls are unnecessary.
Code snapshot (permalink): [https://github.com/apache/hadoop/blob/2a67e2b1a0e3a5f91056f5b977ef9c4c07ba6718/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L4002]
Causes additional getFileStatus call here (permalink): [https://github.com/apache/hadoop/blob/1921e94292f0820985a0cfbf8922a2a1a67fe921/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L308]
Current code snippet:
{code:java}
/**
* Override superclass so as to disable symlink resolution and so avoid
* some calls to the FS which may have problems when the store is being
* inconsistent.
* {@inheritDoc}
*/
@Override
public FileStatus[] globStatus(
final Path pathPattern,
final PathFilter filter)
throws IOException {
entryPoint(INVOCATION_GLOB_STATUS);
return Globber.createGlobber(this)
.withPathPattern(pathPattern)
.withPathFiltern(filter)
.withResolveSymlinks(true)
.build()
.glob();
}
{code}
The fix should be pretty simple, just flip "withResolveSymlinks" to false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org