You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Jens Rabe <ra...@t-online.de> on 2015/04/19 10:21:28 UTC
How to make MiniDFSCluster available in tests in Hadoop's code base?
Hello,
I wanted to write a fix for MAPREDUCE-6320, and for this, I need a mini cluster for the tests. The hadoop-mapreduce-client-core imports the hadoop-hdfs in the „test“ scope, but the MiniDFSCluster is not present there, as it is in the „test“ sources of said module. What grinds my gears is that when starting a new project and adding the hadoop modules as Maven dependencies, the MiniDFSCluster indeed is available. So, what am I doing wrong and how can I get it? Do I have to do an initial compile with a specific profile?
Regards,
Jens
Re: How to make MiniDFSCluster available in tests in Hadoop's code base?
Posted by Jens Rabe <ra...@t-online.de>.
Hello,
thanks for pointing this out. I added this dependency and it works as I expect it to.
> Am 19.04.2015 um 20:02 schrieb Steve Loughran <st...@hortonworks.com>:
>
>
>> On 19 Apr 2015, at 09:21, Jens Rabe <ra...@t-online.de> wrote:
>>
>> Hello,
>>
>> I wanted to write a fix for MAPREDUCE-6320, and for this, I need a mini cluster for the tests. The hadoop-mapreduce-client-core imports the hadoop-hdfs in the „test“ scope, but the MiniDFSCluster is not present there, as it is in the „test“ sources of said module. What grinds my gears is that when starting a new project and adding the hadoop modules as Maven dependencies, the MiniDFSCluster indeed is available. So, what am I doing wrong and how can I get it? Do I have to do an initial compile with a specific profile?
>>
>> Regards,
>> Jens
>
> it's a standalone artifact now
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-minicluster</artifactId>
> <scope>test</scope>
> </dependency>
Re: How to make MiniDFSCluster available in tests in Hadoop's code
base?
Posted by Steve Loughran <st...@hortonworks.com>.
> On 19 Apr 2015, at 09:21, Jens Rabe <ra...@t-online.de> wrote:
>
> Hello,
>
> I wanted to write a fix for MAPREDUCE-6320, and for this, I need a mini cluster for the tests. The hadoop-mapreduce-client-core imports the hadoop-hdfs in the „test“ scope, but the MiniDFSCluster is not present there, as it is in the „test“ sources of said module. What grinds my gears is that when starting a new project and adding the hadoop modules as Maven dependencies, the MiniDFSCluster indeed is available. So, what am I doing wrong and how can I get it? Do I have to do an initial compile with a specific profile?
>
> Regards,
> Jens
it's a standalone artifact now
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<scope>test</scope>
</dependency>