You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2009/09/30 04:06:32 UTC
[jira] Created: (HDFS-663) DFSIO for append
DFSIO for append
----------------
Key: HDFS-663
URL: https://issues.apache.org/jira/browse/HDFS-663
Project: Hadoop HDFS
Issue Type: Sub-task
Components: test
Affects Versions: Append Branch
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
Fix For: Append Branch
Attachments: appendDFSIO.patch
Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763695#action_12763695 ]
Konstantin Shvachko commented on HDFS-663:
------------------------------------------
Hudson is trying to apply this to hdfs trunk, which does not have this code after HDFS-641. So since there were no objections from folks in charge of HDFS-641 I am commiting this to 0.21 and HDFS-641 branches.
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-663:
-------------------------------------
Attachment: appendDFSIO-MR.patch
appendDFSIO.patch
Two separate patches for 0.21 and HDFS-641 branches.
Some changes:
- I made testIO() run on MiniDFSCluster. So now it is really tested on hdfs instead of local file system.
- Changed usage.
- Turned on append in the configuration for the test.
- testIO() prints stats after each of the three jobs.
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-663) DFSIO for append
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760816#action_12760816 ]
Konstantin Boudnik commented on HDFS-663:
-----------------------------------------
+1
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: Append Branch
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: Append Branch
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762774#action_12762774 ]
Konstantin Shvachko commented on HDFS-663:
------------------------------------------
I ran test-core and test-patch against branch 0.21. Both successful.
{code}
. [exec] There appear to be 108 release audit warnings before the patch and 108 release audit warnings after applying the patch.
[exec] +1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
BUILD SUCCESSFUL
Total time: 13 minutes 33 seconds
{code}
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-663:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-663) DFSIO for append
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762852#action_12762852 ]
Hadoop QA commented on HDFS-663:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12420873/appendDFSIO.patch
against trunk revision 822153.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/55/console
This message is automatically generated.
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763861#action_12763861 ]
Konstantin Shvachko commented on HDFS-663:
------------------------------------------
Re-ran test-patch for both branches. No new warnings. Attaching results for the MR branch:
{noformat}
[exec] There appear to be 169 release audit warnings before the patch and 169 release audit warnings after applying the patch.
[exec] +1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
BUILD SUCCESSFUL
Total time: 23 minutes 5 seconds
{noformat}
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO-MR.patch, appendDFSIO.patch, appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-663:
-------------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-663:
-------------------------------------
Attachment: appendDFSIO.patch
This adds append benchmark to TestDFSIO.
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: Append Branch
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: Append Branch
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-663) DFSIO for append
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-663:
-------------------------------------
Affects Version/s: (was: Append Branch)
0.21.0
Fix Version/s: (was: Append Branch)
0.21.0
> DFSIO for append
> ----------------
>
> Key: HDFS-663
> URL: https://issues.apache.org/jira/browse/HDFS-663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: appendDFSIO.patch
>
>
> Create a benchmark for append within the framework of TestDFSIO.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.