[jira] [Comment Edited] (CRUNCH-304) MRPipeline.plan() does not clear up the temporary hadoop- folder it creates

["Ganeshbabu Nelamangala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CRUNCH-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832636#comment-13832636 ] 

Ganeshbabu Nelamangala edited comment on CRUNCH-304 at 11/26/13 3:03 PM:
-------------------------------------------------------------------------

Yes, that should work. If cleanup is available below should work to solve our issue. 
1. generate the DOT diagram for the targets pipeline holds then will 
2. do a cleanup() to clear the targets.
3. call done() which performs cleanup to clear any temporary folder created.
[~mkwhitacre] please suggest if this is good.


was (Author: ganeshbabu):
Yes, that should work. If cleanup is available below should work to solve our issue. 
1. generate the DOT diagram for the targets pipeline holds then will 
2. do a cleanup() to clear the targets.
3. call done() which performs cleanup to clear any temporary folder it creates.
[~mkwhitacre] please suggest if this is good.

> MRPipeline.plan() does not clear up the temporary hadoop-<username> folder it creates
> -------------------------------------------------------------------------------------
>
>                 Key: CRUNCH-304
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-304
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: Hadoop 2.0.0-cdh4.2.1
> Subversion file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.2.1/src/hadoop-common-project/hadoop-common -r 144bd548d481c2774fab2bec2ac2645d190f705b
> Compiled by jenkins on Mon Apr 22 10:26:03 PDT 2013
> From source with checksum aef88defdddfb22327a107fbd7063395
>            Reporter: Ganeshbabu Nelamangala
>            Assignee: Josh Wills
>            Priority: Minor
>              Labels: easyfix, newbie, patch
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> As a user i just want to run MRPipeline.plan() to retrieve the PlanningParameters.PIPELINE_PLAN_DOTFILE for the current pipeline configuration. However it appears that since we don't actually call run(), the Map that is created to hold outputTargets still has items in it, and the cleanup() method will not execute under the condition of having elements in it, to delete the tmp directory that gets created when creating a new MRPipeline object. Since we don't really want to execute the code if we just want to create a plan then I don't know how we can cleanup this Map. Basically the temporary hadoop folders left behind is our problem.



--
This message was sent by Atlassian JIRA
(v6.1#6144)