You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/03 22:33:00 UTC
[jira] [Commented] (MAPREDUCE-7127) Add aggregated webservice
endpoint to fetch all tasks including their attempts
[ https://issues.apache.org/jira/browse/MAPREDUCE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568872#comment-16568872 ]
ASF GitHub Bot commented on MAPREDUCE-7127:
-------------------------------------------
GitHub user bschell opened a pull request:
https://github.com/apache/hadoop/pull/407
MAPREDUCE-7127. Add aggregated webservice endpoints to fetch all tasks & their taskAttempts
There is a usecase to poll Hadoop for various running Tasks and display info on each individual task attempt to the user. To improve performance add additional endpoints to HS and AM webservices that will fetch aggregated data on hadoop tasks including their task attempts for a current job.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bschell/hadoop bschelle/taskservice2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/407.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #407
----
commit 5c314b7fd30c2af803d6099abda22ad2546ba5bb
Author: Scheller <bs...@...>
Date: 2018-07-11T20:56:54Z
Add aggregated webservice endpoints to fetch all tasks including their attempts
There is a usecase to poll Hadoop for various running Tasks and display info on each individual task attempt to the user. To improve performance add additional endpoints to HS and AM webservices that will fetch aggregated data on hadoop tasks including their task attempts for a current job.
----
> Add aggregated webservice endpoint to fetch all tasks including their attempts
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7127
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7127
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Brandon Scheller
> Priority: Major
> Labels: webservices
>
> We have a usecase where we would like to poll Hadoop for various running Tasks and display info on each individual task attempt to the user.
> On large clusters with many tasks, this is an issue because using the current webservice APIs we would have to make many repeat calls to the same task attempts endpoints.
> To improve performance, we have added additional endpoints to HS and AM webservices that will fetch aggregated data on hadoop tasks including their task attempts for a job.
> This would look like
> - taskDescriptions webservice endpoint to AMWebServices
> @Path("/jobs/ \{jobid}/taskDescriptions")
>
> - describeTasks webservice endpoint to HSWebServices
> @Path("/mapreduce/jobs/\{jobid}/describeTasks")
> It would be similar to the current getJobTasks() in HS and AM webservices except it would include each of the task attempts for each task.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org