You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2010/04/01 06:04:27 UTC

[jira] Created: (PIG-1347) Clear up output directory for a failed job

Clear up output directory for a failed job
------------------------------------------

                 Key: PIG-1347
                 URL: https://issues.apache.org/jira/browse/PIG-1347
             Project: Pig
          Issue Type: Bug
          Components: impl
    Affects Versions: 0.7.0
            Reporter: Daniel Dai
            Assignee: Daniel Dai
             Fix For: 0.8.0


FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (PIG-1347) Clear up output directory for a failed job

Posted by "Ashitosh Darbarwar (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashitosh Darbarwar reassigned PIG-1347:
---------------------------------------

    Assignee: Ashitosh Darbarwar  (was: Daniel Dai)

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1347) Clear up output directory for a failed job

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871514#action_12871514 ] 

Daniel Dai commented on PIG-1347:
---------------------------------

In current code, we use StoreFunc.cleanupOnFailure for this purpose. FileLocalizer.deleteOnFail should be removed. So this issue is fixed in trunk, we should remove redundant code.

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1347) Clear up output directory for a failed job

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871862#action_12871862 ] 

Ashutosh Chauhan commented on PIG-1347:
---------------------------------------

Patch is pretty straightforward and harmless as it only removes code and does not add any thing new. Only concern I have is FileLocalizer.registerDeleteOnFail() is a public method so its possible that some one using Pig's java api is using this method to do the cleanup himself previously.  So, this can be considered as backward incompatible change. But, Daniel explained to me that this method was meant for Pig's internal usage and clean up in any case was taken care by Pig before the recent store func changes, so user need not to worry about it. So, its extremely unlikely that someone is using it. 
So, +1 on committing.

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>         Attachments: PIG-1347-1.patch
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (PIG-1347) Clear up output directory for a failed job

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-1347.
-----------------------------

    Hadoop Flags: [Reviewed]
      Resolution: Fixed

Patch committed.

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>         Attachments: PIG-1347-1.patch
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1347) Clear up output directory for a failed job

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-1347:
----------------------------

    Attachment:     (was: PIG-1347-1.patch)

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>         Attachments: PIG-1347-1.patch
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1347) Clear up output directory for a failed job

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-1347:
----------------------------

    Attachment: PIG-1347-1.patch

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>         Attachments: PIG-1347-1.patch
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1347) Clear up output directory for a failed job

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/PIG-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-1347:
----------------------------

    Attachment: PIG-1347-1.patch

Remove redundant code.

> Clear up output directory for a failed job
> ------------------------------------------
>
>                 Key: PIG-1347
>                 URL: https://issues.apache.org/jira/browse/PIG-1347
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Ashitosh Darbarwar
>             Fix For: 0.8.0
>
>         Attachments: PIG-1347-1.patch
>
>
> FileLocalizer.deleteOnFail suppose to track the output files need to be deleted in case the job fails. However, in the current code base, deleteOnFail is dangling. registerDeleteOnFail and triggerDeleteOnFail is called by nobody. We need to bring it back.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.