[jira] Updated: (WHIRR-92) Add a benchmark for Hadoop clusters

["Tom White (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/WHIRR-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated WHIRR-92:
---------------------------

    Attachment: WHIRR-92.patch

Here's a patch for running TeraSort and TestDFSIO (and test of HDFS throughput) benchmarks. There's more work to run this against clusters and tune settings, but this is ready for inclusion, I think.

The version of TestDFSIO needed is the latest from the 0.20 Hadoop branch (since it implements the Tool interface), which can be built with {{ant jar-test}} and copied to services/hadoop/lib.

To run the whole benchmark suite:

{noformat}
mvn verify -Pintegration -DargLine="-Dwhirr.test.identity=... -Dwhirr.test.credential=... -Dconfig=.whirr-test.properties" -Dit.test=BenchmarkSuite
{noformat}

To run a particular test:

{noformat}
mvn verify -Pintegration -DargLine="-Dwhirr.test.identity=... -Dwhirr.test.credential=... -Dconfig=.whirr-test.properties" -Dit.test=HadoopServiceTestDFSIOBenchmark
{noformat}

By default no benchmarks are run when you run integration tests.

> Add a benchmark for Hadoop clusters
> -----------------------------------
>
>                 Key: WHIRR-92
>                 URL: https://issues.apache.org/jira/browse/WHIRR-92
>             Project: Whirr
>          Issue Type: Test
>          Components: service/hadoop
>            Reporter: Tom White
>             Fix For: 0.3.0
>
>         Attachments: WHIRR-92.patch
>
>
> To help tune cluster configurations on various providers we should run and record the results of a benchmark (e.g. Terasort) on Hadoop clusters.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.