You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2013/08/01 02:17:48 UTC
[jira] [Commented] (MAPREDUCE-5193) A few MR tests use block sizes
which are smaller than the default minimum block size
[ https://issues.apache.org/jira/browse/MAPREDUCE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725891#comment-13725891 ]
Lars Hofhansl commented on MAPREDUCE-5193:
------------------------------------------
This just caused us a major head ache with HBase (we used CDH4.3 and our build of HBase, but the same would apply to plain open source HDFS).
There now is an hdfs-site.xml in a jar that is included by HBase, and it overrides any hdfs-site.xml added later to the classpath (as HBase does when adding the Hadoop classpath in order to read in Hadoop's hdfs-site.xml).
We'll fix this in HBase (either by not including the test jars or by changing how we form the classpath), but generally it seems a bad choice to include any hdfs-site or core-site, etc, files in the jars.
> A few MR tests use block sizes which are smaller than the default minimum block size
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5193
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5193
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.1.0-beta
> Reporter: Aaron T. Myers
> Assignee: Andrew Wang
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5156.1.patch, mapreduce-5193-1.patch
>
>
> HDFS-4305 introduced a new configurable minimum block size of 1MB. A few MR tests deliberately set much smaller block sizes. This JIRA is to update those tests to fix these failing tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira