You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Milind Bhandarkar (JIRA)" <ji...@apache.org> on 2007/10/28 20:35:50 UTC
[jira] Created: (HADOOP-2115) Task cwds should be distributed
across partitions
Task cwds should be distributed across partitions
-------------------------------------------------
Key: HADOOP-2115
URL: https://issues.apache.org/jira/browse/HADOOP-2115
Project: Hadoop
Issue Type: Improvement
Components: mapred
Affects Versions: 0.14.3
Environment: All
Reporter: Milind Bhandarkar
Fix For: 0.16.0
Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per physical disk), all tasks of the same job have current working directories that belong to only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads a single disk while other disks may be idle. Idially, each task should get a cwd on different partition. This is related to HADOOP-1991, but emphasizes performance impact.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2115) Task cwds should be distributed
across partitions
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554220 ]
Amareshwari Sri Ramadasu commented on HADOOP-2115:
--------------------------------------------------
bq. I think this was fixed by HADOOP-2227 and we can close this issue.
Yes. It is fixed in 2227.
> Task cwds should be distributed across partitions
> -------------------------------------------------
>
> Key: HADOOP-2115
> URL: https://issues.apache.org/jira/browse/HADOOP-2115
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Fix For: 0.16.0
>
>
> Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per physical disk), all tasks of the same job have current working directories that belong to only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads a single disk while other disks may be idle. Idially, each task should get a cwd on different partition. This is related to HADOOP-1991, but emphasizes performance impact.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2115) Task cwds should be distributed
across partitions
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12553998 ]
Konstantin Shvachko commented on HADOOP-2115:
---------------------------------------------
I think this was fixed by HADOOP-2227 and we can close this issue.
Amareshwari, could you please confirm?
> Task cwds should be distributed across partitions
> -------------------------------------------------
>
> Key: HADOOP-2115
> URL: https://issues.apache.org/jira/browse/HADOOP-2115
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Fix For: 0.16.0
>
>
> Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per physical disk), all tasks of the same job have current working directories that belong to only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads a single disk while other disks may be idle. Idially, each task should get a cwd on different partition. This is related to HADOOP-1991, but emphasizes performance impact.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-2115) Task cwds should be distributed
across partitions
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko resolved HADOOP-2115.
-----------------------------------------
Resolution: Duplicate
Closing this issue since it has already been fixed.
> Task cwds should be distributed across partitions
> -------------------------------------------------
>
> Key: HADOOP-2115
> URL: https://issues.apache.org/jira/browse/HADOOP-2115
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Konstantin Shvachko
> Fix For: 0.16.0
>
>
> Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per physical disk), all tasks of the same job have current working directories that belong to only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads a single disk while other disks may be idle. Idially, each task should get a cwd on different partition. This is related to HADOOP-1991, but emphasizes performance impact.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2115) Task cwds should be distributed
across partitions
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko reassigned HADOOP-2115:
-------------------------------------------
Assignee: Konstantin Shvachko
> Task cwds should be distributed across partitions
> -------------------------------------------------
>
> Key: HADOOP-2115
> URL: https://issues.apache.org/jira/browse/HADOOP-2115
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Konstantin Shvachko
> Fix For: 0.16.0
>
>
> Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per physical disk), all tasks of the same job have current working directories that belong to only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads a single disk while other disks may be idle. Idially, each task should get a cwd on different partition. This is related to HADOOP-1991, but emphasizes performance impact.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.