You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hong Tang (JIRA)" <ji...@apache.org> on 2010/05/05 00:50:03 UTC

[jira] Created: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Allow TestDFSIO to process multiple files for each map
------------------------------------------------------

                 Key: HDFS-1128
                 URL: https://issues.apache.org/jira/browse/HDFS-1128
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Hong Tang


DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hong Tang updated HDFS-1128:
----------------------------

    Attachment: HDFS-1128-yhadoop-20.1xx.patch

patch for yahoo hadoop branch, not to be committed.

> Allow TestDFSIO to process multiple files for each map
> ------------------------------------------------------
>
>                 Key: HDFS-1128
>                 URL: https://issues.apache.org/jira/browse/HDFS-1128
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hong Tang
>         Attachments: HDFS-1128-yhadoop-20.1xx.patch
>
>
> DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12868355#action_12868355 ] 

Konstantin Shvachko commented on HDFS-1128:
-------------------------------------------

{{WriteMapper.doIO()}} still creates only one file per map. I probably missed something.
Also in your patch "Average IO rate" and "std deviation" are printed twice.

> Allow TestDFSIO to process multiple files for each map
> ------------------------------------------------------
>
>                 Key: HDFS-1128
>                 URL: https://issues.apache.org/jira/browse/HDFS-1128
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hong Tang
>         Attachments: HDFS-1128-yhadoop-20.1xx-2.patch, HDFS-1128-yhadoop-20.1xx.patch
>
>
> DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hong Tang updated HDFS-1128:
----------------------------

    Attachment: HDFS-1128-yhadoop-20.1xx-2.patch

New patch that addresses Hairong's comments.

> Allow TestDFSIO to process multiple files for each map
> ------------------------------------------------------
>
>                 Key: HDFS-1128
>                 URL: https://issues.apache.org/jira/browse/HDFS-1128
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hong Tang
>         Attachments: HDFS-1128-yhadoop-20.1xx-2.patch, HDFS-1128-yhadoop-20.1xx.patch
>
>
> DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864421#action_12864421 ] 

Hairong Kuang commented on HDFS-1128:
-------------------------------------

The patch looks good except that the javadoc needs to be updated as well.

> Allow TestDFSIO to process multiple files for each map
> ------------------------------------------------------
>
>                 Key: HDFS-1128
>                 URL: https://issues.apache.org/jira/browse/HDFS-1128
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hong Tang
>         Attachments: HDFS-1128-yhadoop-20.1xx.patch
>
>
> DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HDFS-1128) Allow TestDFSIO to process multiple files for each map

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864440#action_12864440 ] 

Hairong Kuang commented on HDFS-1128:
-------------------------------------

+1

> Allow TestDFSIO to process multiple files for each map
> ------------------------------------------------------
>
>                 Key: HDFS-1128
>                 URL: https://issues.apache.org/jira/browse/HDFS-1128
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hong Tang
>         Attachments: HDFS-1128-yhadoop-20.1xx-2.patch, HDFS-1128-yhadoop-20.1xx.patch
>
>
> DFSIO only processes one file in each map, it would be nice to enhance it to process multiple files.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.