You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2009/09/30 04:06:32 UTC

[jira] Created: (HDFS-663) DFSIO for append

DFSIO for append
----------------

                 Key: HDFS-663
                 URL: https://issues.apache.org/jira/browse/HDFS-663
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: test
    Affects Versions: Append Branch
            Reporter: Konstantin Shvachko
            Assignee: Konstantin Shvachko
             Fix For: Append Branch
         Attachments: appendDFSIO.patch

Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763695#action_12763695 ] 

Konstantin Shvachko commented on HDFS-663:
------------------------------------------

Hudson is trying to apply this to hdfs trunk, which does not have this code after HDFS-641. So since there were no objections from folks in charge of HDFS-641 I am commiting this to 0.21 and HDFS-641 branches.

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HDFS-663:
-------------------------------------

    Attachment: appendDFSIO-MR.patch
                appendDFSIO.patch

Two separate patches for 0.21 and HDFS-641 branches.
Some changes:
- I made testIO() run on MiniDFSCluster. So now it is really tested on hdfs  instead of local file system.
- Changed usage.
- Turned on append in the configuration for the test.
- testIO()  prints stats after each of the three jobs. 

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-663) DFSIO for append

Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760816#action_12760816 ] 

Konstantin Boudnik commented on HDFS-663:
-----------------------------------------

+1

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: Append Branch
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: Append Branch
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762774#action_12762774 ] 

Konstantin Shvachko commented on HDFS-663:
------------------------------------------

I ran test-core and test-patch against branch 0.21. Both successful.
{code}
.    [exec] There appear to be 108 release audit warnings before the patch and 108 release audit warnings after applying the patch.
     [exec] +1 overall.  
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
     [exec] ======================================================================
     [exec] ======================================================================
     [exec]     Finished build.
     [exec] ======================================================================
     [exec] ======================================================================
BUILD SUCCESSFUL
Total time: 13 minutes 33 seconds
{code}

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HDFS-663:
-------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this.

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-663) DFSIO for append

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762852#action_12762852 ] 

Hadoop QA commented on HDFS-663:
--------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12420873/appendDFSIO.patch
  against trunk revision 822153.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    -1 patch.  The patch command could not apply the patch.

Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/55/console

This message is automatically generated.

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763861#action_12763861 ] 

Konstantin Shvachko commented on HDFS-663:
------------------------------------------

Re-ran test-patch for both branches. No new warnings. Attaching results for the MR branch:
{noformat}
     [exec] There appear to be 169 release audit warnings before the patch and 169 release audit warnings after applying the patch.
     [exec] +1 overall.  
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
     [exec] ======================================================================
     [exec] ======================================================================
     [exec]     Finished build.
     [exec] ======================================================================
     [exec] ======================================================================
BUILD SUCCESSFUL
Total time: 23 minutes 5 seconds
{noformat}

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HDFS-663:
-------------------------------------

    Hadoop Flags: [Reviewed]
          Status: Patch Available  (was: Open)

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HDFS-663:
-------------------------------------

    Attachment: appendDFSIO.patch

This adds append benchmark to TestDFSIO.

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: Append Branch
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: Append Branch
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-663) DFSIO for append

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HDFS-663:
-------------------------------------

    Affects Version/s:     (was: Append Branch)
                       0.21.0
        Fix Version/s:     (was: Append Branch)
                       0.21.0

> DFSIO for append
> ----------------
>
>                 Key: HDFS-663
>                 URL: https://issues.apache.org/jira/browse/HDFS-663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.21.0
>
>         Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.