You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2013/06/11 06:53:20 UTC

[jira] [Created] (MAPREDUCE-5320) Cluster.close() is not called by mapreduce.Job

Josh Rosen created MAPREDUCE-5320:
-------------------------------------

             Summary: Cluster.close() is not called by mapreduce.Job
                 Key: MAPREDUCE-5320
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5320
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 2.0.4-alpha
            Reporter: Josh Rosen
            Assignee: Chris Douglas


For MapReduce jobs run through mapreduce.Job.waitForCompletion(), Cluster.close() never seems to get called.

I noticed this problem in my own customized MapReduce environment, which uses a custom ClientProtocolProvider.  My provider creates ClientProtocol instances that contain resources that need to be cleaned up, such as thread pools.

Cluster.close() calls ClientProtocolProvider.close(ClientProtocol), which should clean up these resources.  Cluster.close() is called by mapred.JobClient code and by the CLI, but it doesn't seem to get called for jobs started through mapred.Job.waitForCompletion().  This leads to resource leaks in my application.

This seems like a bug, since mapreduce.Job creates a resource (the Cluster instance) that it never cleans up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira