You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2010/07/09 19:53:50 UTC
[jira] Resolved: (PIG-138) Support for Multiple tmp directories and
clean up
[ https://issues.apache.org/jira/browse/PIG-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-138.
--------------------------------
Resolution: Won't Fix
Not clear that we need it and it will add complexity to the code
> Support for Multiple tmp directories and clean up
> -------------------------------------------------
>
> Key: PIG-138
> URL: https://issues.apache.org/jira/browse/PIG-138
> Project: Pig
> Issue Type: Improvement
> Environment: Pig Local
> Reporter: Amir Youssefi
> Assignee: Pi Song
>
> This is to separate additional improvements and original requirement on PIG-129 issue.
> Pasting comments of Pi Song.
> "I think the concept of multi-dir temp file creator (LocalDirAllocator in Hadoop) should be adopted to Pig. What it does is:-
> * You can set up a set of tmp file dirs in configuration (They can be on different physical drives so you can utilize more disk space)
> * When a temp file is being created, the system will probe the given temp dirs in round-robin fashion
> * For a selected temp dir, if it exists and you have permission to write, temp file will be created
> * For a selected temp dir, it it doesn't exist or you don't have permission to write, the temp dir will be kept in the black list, thus not being used later on.
> * For the next temp file, move on to the next temp dir
> "
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.