You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@accumulo.apache.org by Amit Kapoor <am...@koanect.com> on 2014/09/25 23:41:58 UTC

MapReduce Integration Tests for Accumulo 1.5.1

Hi folks,

  I have not been able to find anything useful on the web, so hoping someone here has an example
  they can share. We are currently using Accumulo 1.5.1 and have quite a few map-reduce jobs that
  we need to start having integration tests for. My question is does anyone have a small example of
  integration test for map-reduce using minicluster (or something similar) that we can incorporate
  into our continuous deployment?

  I noticed a MapReduceIT in 1.6.0 source code, but unfortunately we cannot use 1.6.0 till
  some other open source packages we depend on upgrade to that.

  regards

Amit

Re: MapReduce Integration Tests for Accumulo 1.5.1

Posted by Amit Kapoor <am...@koanect.com>.

Hi Josh,

  Thanks. We will take a look.

  regards

Amit
On Thu, Sep 25, 2014 at 08:14:44PM -0400, Josh Elser wrote:
> Have you looked at the MiniMRClusterFactory and MiniMRClientCluster yet? Hadoop has some examples
> which start the MiniMR cluster and then use ToolRunner to submit a job.
> 
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestLargeSort.java
>

Re: MapReduce Integration Tests for Accumulo 1.5.1

Posted by Josh Elser <jo...@gmail.com>.

Have you looked at the MiniMRClusterFactory and MiniMRClientCluster yet? Hadoop has some examples which start the MiniMR cluster and then use ToolRunner to submit a job.

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestLargeSort.java

Amit Kapoor wrote:
> Hi folks,
>
>    I have not been able to find anything useful on the web, so hoping someone here has an example
>    they can share. We are currently using Accumulo 1.5.1 and have quite a few map-reduce jobs that
>    we need to start having integration tests for. My question is does anyone have a small example of
>    integration test for map-reduce using minicluster (or something similar) that we can incorporate
>    into our continuous deployment?
>
>    I noticed a MapReduceIT in 1.6.0 source code, but unfortunately we cannot use 1.6.0 till
>    some other open source packages we depend on upgrade to that.
>
>    regards
>
> Amit