[jira] Updated: (PIG-1265) Change LoadMetadata and StoreMetadata to use Job instead of Configuraiton and add a cleanupOnFailure method to StoreFuncInterface

["Pradeep Kamath (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pradeep Kamath updated PIG-1265:
--------------------------------

    Assignee: Pradeep Kamath  (was: Pradeep Kamath)
      Status: Patch Available  (was: Open)

> Change LoadMetadata and StoreMetadata to use Job instead of Configuraiton and add a cleanupOnFailure method to StoreFuncInterface
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1265
>                 URL: https://issues.apache.org/jira/browse/PIG-1265
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>             Fix For: 0.7.0
>
>         Attachments: PIG-1265.patch
>
>
> Speaking to the hadoop team folks, the direction in hadoop is to use Job instead of Configuration - for example InputFormat/OutputFormat implementations use Job to store input/output location. So pig should also do the same in LoadMetadata and StoreMetadata to be closer to hadoop.
> Currently when a job fails, pig assumes the output locations (corresponding to the stores in the job) are hdfs locations and attempts to delete them. Since output locations could be non hdfs locations, this cleanup should be delegated to the StoreFuncInterface implementation - hence a new method - cleanupOnFailure() should be introduced in StoreFuncInterface and a default implementation should be provided in the StoreFunc abstract class which checks if the location exists on hdfs and deletes it if so.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.