You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Bennie Schut (JIRA)" <ji...@apache.org> on 2010/08/24 11:46:18 UTC

[jira] Created: (HIVE-1587) Files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed

Files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed
--------------------------------------------------------------------------------------------------------------------

                 Key: HIVE-1587
                 URL: https://issues.apache.org/jira/browse/HIVE-1587
             Project: Hadoop Hive
          Issue Type: Bug
          Components: Query Processor
    Affects Versions: 0.7.0
            Reporter: Bennie Schut


I seem to have a lot of files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed.
These jobs ran without any problems.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1587) Files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed

Posted by "Bennie Schut (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901802#action_12901802 ] 

Bennie Schut commented on HIVE-1587:
------------------------------------

This might not be the problem but looking at why this happends I noticed the delete used to happen on ExecDriver.execute but seems to have moved to ctx.clear() which is now only executed when the Context was removed.
Is it correct the Driver remains alive for the duration of a connection (when using HiveServer)? Since we use connection pooling to connect to the hive server it means some connections never really close so that would explian what I'm seeing.


r957035

    } finally {
      Utilities.clearMapRedWork(job);
      try {
        emptyScratchDir.getFileSystem(job).delete(emptyScratchDir, true);
        if (returnVal != 0 && rj != null) {
          rj.killJob();
        }
        runningJobKillURIs.remove(rj.getJobID());
      } catch (Exception e) {
      }
    }


r982490

    } finally {
      Utilities.clearMapRedWork(job);
      try {
        if(ctxCreated)
          ctx.clear();

        if (returnVal != 0 && rj != null) {
          rj.killJob();
        }
        runningJobKillURIs.remove(rj.getJobID());
      } catch (Exception e) {
      }
    }


> Files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-1587
>                 URL: https://issues.apache.org/jira/browse/HIVE-1587
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.7.0
>            Reporter: Bennie Schut
>
> I seem to have a lot of files on /tmp/hive-{username}/hive_{some timestamp}/ which are leftovers from old jobs which never are never removed.
> These jobs ran without any problems.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.