You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2017/09/14 22:17:00 UTC
[jira] [Updated] (MAPREDUCE-6958) Shuffle audit logger should log
size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-6958:
----------------------------------
Attachment: MAPREDUCE-6958.001.patch
Patch that adds the content length to the audit log.
> Shuffle audit logger should log size of shuffle transfer
> --------------------------------------------------------
>
> Key: MAPREDUCE-6958
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Minor
> Attachments: MAPREDUCE-6958.001.patch
>
>
> The shuffle audit logger currently logs the job ID and reducer ID but nothing about the size of the requested transfer. It calculates this as part of the HTTP response headers, so it would be trivial to log the response size. This would be very valuable for debugging network traffic storms from the shuffle handler.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org