You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2010/04/01 07:47:27 UTC

[jira] Created: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

TaskRunner.prepare() and close() can be removed
-----------------------------------------------

                 Key: MAPREDUCE-1662
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
    Affects Versions: 0.22.0
            Reporter: Amareshwari Sriramadasu
             Fix For: 0.22.0


TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated MAPREDUCE-1662:
-------------------------------------

          Status: Resolved  (was: Patch Available)
    Hadoop Flags: [Reviewed]
        Assignee: Amareshwari Sriramadasu
      Resolution: Fixed

+1

I committed this. Thanks, Amareshwari!

> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852564#action_12852564 ] 

Hadoop QA commented on MAPREDUCE-1662:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12440478/patch-1662.txt
  against trunk revision 929712.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/87/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/87/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/87/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/87/console

This message is automatically generated.

> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870069#action_12870069 ] 

Hudson commented on MAPREDUCE-1662:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk #324 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/324/])
    MAPREDUCE-1662. Remove unused methods from TaskRunner. Contributed by Amareshwari Sriramadasu


> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853316#action_12853316 ] 

Amareshwari Sriramadasu commented on MAPREDUCE-1662:
----------------------------------------------------

bq. -1 tests included. 
No new tests added because patch removes dead code.

bq. -1 core tests.
The test failure TestMapReduceLazyOutput.testLazyOutput is not related to the patch. The test failed because of ZipExceptions (MAPREDUCE-1642).
The same test passed on my machine.


> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sriramadasu updated MAPREDUCE-1662:
-----------------------------------------------

    Attachment: patch-1662.txt

Patch removes TaskRunner.prepare(). 
ReduceTaskRunner.close() sets a status "closed" on the Progress object. So, patch does not remove TaskRunner.close(), but removes mapOutputFile.removeAll() from close() methods in MapTaskRunner and ReduceTaskRunner.

> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1662) TaskRunner.prepare() and close() can be removed

Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sriramadasu updated MAPREDUCE-1662:
-----------------------------------------------

    Status: Patch Available  (was: Open)

> TaskRunner.prepare() and close() can be removed
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1662
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1662
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1662.txt
>
>
> TaskRunner.prepare() and close() methods call only mapOutputFile.removeAll(). The removeAll() call is a always a no-op in prepare(), because the directory is always empty during start up of the task. The removeAll() call in close() is useless, because it is followed by a attempt directory cleanup. Since the map output files are in attempt directory,  the call to close() is useless.
> After MAPREDUCE-842, these calls are under TaskTracker space, passing the wrong conf. Now, the calls do not make sense at all.
> I think we can remove the methods.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.