You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Scott Chen (JIRA)" <ji...@apache.org> on 2010/01/15 21:54:54 UTC

[jira] Created: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

MRAsyncDiscService should tolerate missing local.dir
----------------------------------------------------

                 Key: MAPREDUCE-1382
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Scott Chen
            Assignee: Zheng Shao


Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Dan Adkins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dan Adkins updated MAPREDUCE-1382:
----------------------------------

    Attachment: MAPREDUCE-1382.4.patch

s/finally/catch (Exception e)/

The previous patch was still throwing the exception instead of ignoring it as intended.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907441#action_12907441 ] 

Zheng Shao commented on MAPREDUCE-1382:
---------------------------------------

I believe other logic in TaskTracker/JobTracker will fail and report in that case.


> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Dan Adkins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907494#action_12907494 ] 

Dan Adkins commented on MAPREDUCE-1382:
---------------------------------------

I have applied the latest patch to trunk and *still* I cannot start a task tracker if the the directory does not exist:

10/09/09 00:36:29 WARN mapred.TaskTracker: Task Tracker local can not create directory: /mnt/data8/mr20
10/09/09 00:36:29 WARN util.MRAsyncDiskService: Cannot create toBeDeleted in /mnt/data8/mr20. Ignored.
10/09/09 00:36:29 ERROR mapred.TaskTracker: Can not start task tracker because java.io.FileNotFoundException: File /mnt/data8/mr20/toBeDeleted does not exist.
	at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:301)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1071)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1096)
	at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
	at org.apache.hadoop.mapreduce.util.MRAsyncDiskService.<init>(MRAsyncDiskService.java:103)
	at org.apache.hadoop.mapreduce.util.MRAsyncDiskService.<init>(MRAsyncDiskService.java:127)
	at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:616)
	at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1323)
	at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3508)


> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated MAPREDUCE-1382:
----------------------------------

    Status: Patch Available  (was: Open)

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated MAPREDUCE-1382:
----------------------------------

    Attachment: MAPREDUCE-1382.2.patch

Todd, you are right. LocalFileSystem will throw IOE in that case.
This patch addresses Todd's concern.


> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907812#action_12907812 ] 

Scott Chen commented on MAPREDUCE-1382:
---------------------------------------

The try { } block in cleanupAllVolumes() also needs the fix Dan pointed out.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated MAPREDUCE-1382:
----------------------------------

    Attachment: MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch

patch for 0.20

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907439#action_12907439 ] 

Scott Chen commented on MAPREDUCE-1382:
---------------------------------------

Does it make sense to throw IOE if all volumes in the constructor are missing?

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803467#action_12803467 ] 

Todd Lipcon commented on MAPREDUCE-1382:
----------------------------------------

Hi Zheng,

I wonder whether localFileSystem.listStatus will ever throw IOE in the case of a bad volume? Yours handles the case where the volume doesn't exist at all, but I can imagine if it's on a broken mount, the listStatus will get an I/O Error from the underlying filesystem and throw at that point. Do you need to catch this and handle in the same way as if it were null? Thanks.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated MAPREDUCE-1382:
----------------------------------

    Attachment: MAPREDUCE-1382.1.patch

Added a test that tests deletion non-existing files.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907814#action_12907814 ] 

Scott Chen commented on MAPREDUCE-1382:
---------------------------------------

Zheng, Is it possible that you can add a simple test for this bug? Use some non-exist volume name to construct a MRAsyncDiskService.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907489#action_12907489 ] 

Scott Chen commented on MAPREDUCE-1382:
---------------------------------------

+1, the patch looks good to me

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir

Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated MAPREDUCE-1382:
----------------------------------

    Attachment: MAPREDUCE-1382.3.patch

Fixed unit test.

> MRAsyncDiscService should tolerate missing local.dir
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1382
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Scott Chen
>            Assignee: Zheng Shao
>         Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch
>
>
> Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.