testing and unit testing pig scripts.

[Paul Rogers <pa...@shaw.ca>]

Can any one give advice on testing the output of a hadoop system or unit
testing for pig scripts?

The best I have so far is to create a minimal data set and run that through
the the system, and check the output. Id much prefer to be able to unit test
the scripts.

Thanks

Paul

Re: testing and unit testing pig scripts.

[Bill Graham <bi...@gmail.com>]

We unit test pig scripts in local mode by doing the following from JUnit:

- Create an instance of PigServer in local mode
- Read a script line by line while doing some basic token replacement for
input/output path location before calling registerQuery.
- Execute the script using test data on the local filesystem as input
- Assert the contents of the test output match another file found on the
local system

hth,
Bill


On Thu, Mar 18, 2010 at 2:00 PM, Paul Rogers <pa...@shaw.ca> wrote:

> Can any one give advice on testing the output of a hadoop system or unit
> testing for pig scripts?
>
> The best I have so far is to create a minimal data set and run that through
> the the system, and check the output. Id much prefer to be able to unit
> test
> the scripts.
>
> Thanks
>
> Paul
>