pipeline.done with pcollection.materialize in tests, why ?

[Rahul <rs...@xebia.com>]

Hi,

I have seen tests where /Pipeline.done()/ was being called in the 
/@After/ method  of a junit or at the end of test after all assertions 
have been done.
All of these tests were using the /materialize() /API to get the 
collection data.
Now, why is /pipeline.done()/  is being called in such cases ? Does it 
close some open ends, if so then which ones ?

regards
Rahul

Re: pipeline.done with pcollection.materialize in tests, why ?

[Josh Wills <jw...@cloudera.com>]

Pipeline.done does some cleanup, like deleting the temp directories that
contain any intermediate output Crunch generated, including the
materialized data. With the changes in CRUNCH-24 to move all of the
temporary directories to use JUnit's TemporaryFolder, I don't think the
done() calls are necessary, but they aren't doing any harm.

On Thu, Jul 26, 2012 at 4:27 AM, Rahul <rs...@xebia.com> wrote:

> Hi,
>
> I have seen tests where /Pipeline.done()/ was being called in the /@After/
> method  of a junit or at the end of test after all assertions have been
> done.
> All of these tests were using the /materialize() /API to get the
> collection data.
> Now, why is /pipeline.done()/  is being called in such cases ? Does it
> close some open ends, if so then which ones ?
>
> regards
> Rahul
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>