You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2013/08/01 02:17:48 UTC

[jira] [Commented] (MAPREDUCE-5193) A few MR tests use block sizes which are smaller than the default minimum block size

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725891#comment-13725891 ] 

Lars Hofhansl commented on MAPREDUCE-5193:
------------------------------------------

This just caused us a major head ache with HBase (we used CDH4.3 and our build of HBase, but the same would apply to plain open source HDFS).
There now is an hdfs-site.xml in a jar that is included by HBase, and it overrides any hdfs-site.xml added later to the classpath (as HBase does when adding the Hadoop classpath in order to read in Hadoop's hdfs-site.xml).

We'll fix this in HBase (either by not including the test jars or by changing how we form the classpath), but generally it seems a bad choice to include any hdfs-site or core-site, etc, files in the jars.

                
> A few MR tests use block sizes which are smaller than the default minimum block size
> ------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5193
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5193
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 2.1.0-beta
>            Reporter: Aaron T. Myers
>            Assignee: Andrew Wang
>             Fix For: 2.1.0-beta
>
>         Attachments: MAPREDUCE-5156.1.patch, mapreduce-5193-1.patch
>
>
> HDFS-4305 introduced a new configurable minimum block size of 1MB. A few MR tests deliberately set much smaller block sizes. This JIRA is to update those tests to fix these failing tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira