[Hadoop Wiki] Update of "TestingNov2009" by SteveLoughran

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "TestingNov2009" page has been changed by SteveLoughran.
The comment on this change is: Started page on testing.
http://wiki.apache.org/hadoop/TestingNov2009

--------------------------------------------------

New page:
= Nov 2009 Testing Framework Conference Call =

Some of the people on the Hadoop list are organising a quick conference call on the topic of testing, this wiki page is to go with it

== JIRA Issues ==
 * [[https://issues.apache.org/jira/browse/HADOOP-6332 |HADOOP-6332]] Large-scale Automated Test Framework
 * [[https://issues.apache.org/jira/browse/HADOOP-6248 | HADOOP-6248]] Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
 * [[https://issues.apache.org/jira/browse/HDFS-708 | HDFS-708 ]]: A stress-test tool for HDFS.

= Use Cases =

Here are some of the use cases that come up when you consider testing Hadoop

== Benchmarking ==

One use case that comes up is stress testing clusters; to see the cluster supports Hadoop "as well as it should", and trying to find out why it doesn't, if it is not adequate. What we have today is [[Terasort]], where you have to guess the approximate numbers then run the job. Terasort creates its own test data, which is good, but it doesn't stress the CPUs as realistically as many workloads, and it generates lots of intermediate and final data; there is no reduction.

 * [[http://www.slideshare.net/steve_l/benchmarking-1840029 | Benchmarking slides]]

== Basic Cluster Health Tests ==

There are currently no tests that work with Hadoop via the web pages, no job submission and monitoring. It is in fact possible to bring up a Hadoop cluster in which JSP doesn't work, but the basic tests all appear well -even including TeraSort, provided you use the low-level APIs

== Testing underlying platforms ==

We need to test the underlying platforms, from the JVM and Linux distributions to any Infrastructure-on-Demand APIs that provide VMs on demand, machines which can run Hadoop.

== Exploring the Hadoop Configuration Space ==

There are a lot of Hadoop configuration options, even ignoring those of the underlying machines and network.

== Testing applications that run on Hadoop ==

This was goal of Alex's Circus prototype: something to make it easier for you to be confident that your code will work.

== Testing changes to Hadoop, fast ==

Hadoop unit/functional testing is slow with MiniMR/MiniDFS cluster setup and teardowns per test. This could be addressed by having more Mini cluster reuse, but it could be even faster if people could push out newly compiled JARs and test them at scale.

== Testing Hadoop Distributions ==

This is a problem which Cloudera and others who distribute/internally package and deploy Hadoop have: you need to know that your RPMs or other redistributables work.

It's similar to the cluster acceptance test problem, except that you need to create the distribution packages and install them on the remote machines, then run the tests.