You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Nishan Shetty (JIRA)" <ji...@apache.org> on 2012/05/28 17:11:23 UTC
[jira] [Created] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and
ClusterStatus.getReduceTasks() is giving one when no job is running
Nishan Shetty created MAPREDUCE-4288:
----------------------------------------
Summary: ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
Key: MAPREDUCE-4288
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0-alpha
Reporter: Nishan Shetty
When no job is running in the cluster invoke the ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() API's
Observed that these API's are returning one instead of zero(as no job is running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and
ClusterStatus.getReduceTasks() is giving one when no job is running
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291555#comment-13291555 ]
Devaraj K commented on MAPREDUCE-4288:
--------------------------------------
It is coming due to the hard-coded values in the below code,
{code:title=ResourceMgrDelegate.java|borderStyle=solid}
public ClusterMetrics getClusterMetrics() throws IOException,
InterruptedException {
GetClusterMetricsRequest request = recordFactory.newRecordInstance(GetClusterMetricsRequest.class);
GetClusterMetricsResponse response = applicationsManager.getClusterMetrics(request);
YarnClusterMetrics metrics = response.getClusterMetrics();
ClusterMetrics oldMetrics = new ClusterMetrics(1, 1, 1, 1, 1, 1,
metrics.getNumNodeManagers() * 10, metrics.getNumNodeManagers() * 2, 1,
metrics.getNumNodeManagers(), 0, 0);
return oldMetrics;
}
{code}
Here we cannot get runningMaps, runningReduces, occupiedMapSlots...etc from RM because the yarn cluster is completely based on the resources and resource usages.
It doesn't look good to show these hard-coded values always to the user when they try to get cluster status using the JobClient.getClusterStatus() API.
Any thoughts on this?
> ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Nishan Shetty
>
> When no job is running in the cluster invoke the ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() API's
> Observed that these API's are returning one instead of zero(as no job is running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and
ClusterStatus.getReduceTasks() is giving one when no job is running
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla reassigned MAPREDUCE-4288:
-------------------------------------------
Assignee: Karthik Kambatla
> ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Nishan Shetty
> Assignee: Karthik Kambatla
>
> When no job is running in the cluster invoke the ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() API's
> Observed that these API's are returning one instead of zero(as no job is running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and
ClusterStatus.getReduceTasks() is giving one when no job is running
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396130#comment-13396130 ]
Karthik Kambatla commented on MAPREDUCE-4288:
---------------------------------------------
In YARN, the ClusterMetrics should only correspond to numNodeManagers, numActiveJobs(), numActiveContainers(), availableResources(). Other job/app-specific metrics should move to the corresponding AMs. JobStatus would be a good place to have these metrics.
Subsequently, JobClient.getClusterStatus() can correspond to the job-specific metrics (would be a misnomer).
Comments?
> ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Nishan Shetty
> Assignee: Karthik Kambatla
>
> When no job is running in the cluster invoke the ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() API's
> Observed that these API's are returning one instead of zero(as no job is running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and
ClusterStatus.getReduceTasks() is giving one when no job is running
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla reassigned MAPREDUCE-4288:
-------------------------------------------
Assignee: (was: Karthik Kambatla)
Couldn't get around to this, marking it as unassigned should anyone be interested.
Will pick it up again if still available later.
> ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Nishan Shetty
>
> When no job is running in the cluster invoke the ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() API's
> Observed that these API's are returning one instead of zero(as no job is running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira