[jira] Commented: (HADOOP-4422) S3 file systems should not create bucket

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650020#action_12650020 ] 

Hadoop QA commented on HADOOP-4422:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12394458/hadoop-s3n-nocreate.patch
  against trunk revision 719787.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no tests are needed for this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3638/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3638/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3638/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3638/console

This message is automatically generated.

> S3 file systems should not create bucket
> ----------------------------------------
>
>                 Key: HADOOP-4422
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4422
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 0.18.1
>            Reporter: David Phillips
>            Assignee: David Phillips
>         Attachments: hadoop-s3n-nocreate.patch, hadoop-s3n-nocreate.patch
>
>
> Both S3 file systems (s3 and s3n) try to create the bucket at every initialization.  This is bad because
> * Every S3 operation costs money.  These unnecessary calls are an unnecessary expense.
> * These calls can fail when called concurrently.  This makes the file system unusable in large jobs.
> * Any operation, such as a "fs -ls", creates a bucket.  This is counter-intuitive and undesirable.
> The initialization code should assume the bucket exists:
> * Creating a bucket is a very rare operation.  Accounts are limited to 100 buckets.
> * Any check at initialization for bucket existence is a waste of money.
> Per Amazon: "Because bucket operations work against a centralized, global resource space, it is not appropriate to make bucket create or delete calls on the high availability code path of your application. It is better to create or delete buckets in a separate initialization or setup routine that you run less often."

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.