[GitHub] [hadoop] steveloughran commented on a change in pull request #1601: HADOOP-16635. S3A innerGetFileStatus scans for directories-only still does a HEAD.

[GitBox <gi...@apache.org>]

steveloughran commented on a change in pull request #1601: HADOOP-16635. S3A innerGetFileStatus scans for directories-only still does a HEAD.
URL: https://github.com/apache/hadoop/pull/1601#discussion_r333942954
 
 

 ##########
 File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
 ##########
 @@ -2730,39 +2730,41 @@ S3AFileStatus innerGetFileStatus(final Path f,
    * @throws FileNotFoundException when the path does not exist
    * @throws IOException on other problems.
    */
+  @VisibleForTesting
   @Retries.RetryTranslated
-  private S3AFileStatus s3GetFileStatus(final Path path,
-      String key,
+  S3AFileStatus s3GetFileStatus(final Path path,
+      final String key,
       final Set<StatusProbeEnum> probes,
       final Set<Path> tombstones) throws IOException {
-    if (!key.isEmpty() && probes.contains(StatusProbeEnum.Head)) {
-      try {
-        ObjectMetadata meta = getObjectMetadata(key);
-
-        if (objectRepresentsDirectory(key, meta.getContentLength())) {
-          LOG.debug("Found exact file: fake directory");
-          return new S3AFileStatus(Tristate.TRUE, path, username);
-        } else {
-          LOG.debug("Found exact file: normal file");
+    if (!key.isEmpty()) {
+      if (probes.contains(StatusProbeEnum.Head) && !key.endsWith("/")) {
 
 Review comment:
   yes. That's exactly my thought. 
   note: none of this API is public, its for avoiding problems on ops where we don't want to look for a file but do for a dir marker. And AFAIK, you can't go from a Path to a / as we strip that off.
   
   How about
   1.  I do a review of all places where we don't ask for the HEAD? So far I think I'm only doing it in create
   1. I clarify in javadocs

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org