You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/10/03 15:56:00 UTC
[jira] [Created] (MAPREDUCE-6974) Add standard configuration keys
for Htrace values, propagate across to MR committers if set
Steve Loughran created MAPREDUCE-6974:
-----------------------------------------
Summary: Add standard configuration keys for Htrace values, propagate across to MR committers if set
Key: MAPREDUCE-6974
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6974
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: job submission, performance
Affects Versions: 3.0.0-beta1
Reporter: Steve Loughran
Priority: Minor
HDFS &c support HTrace logging; HBase sets up spans.
What doesn't do spans is MR jobs or other frameworks with use the MR committers.
That can be addressed by defining some standard configuration keys for HTrace hi/low numbers, setting them in job submission, then passing them over the wire in the Configuration, where setupJob and setupTask can extract them & use for the tracing span. They could also add their own span for taskCommit and jobCommit for performance measurement there.
Although the core code would be in MR, I'd propose putting the keys into Hadoop common,
with some code in {{org.apache.hadoop.tracing.TraceUtils}}, to set it up. That way its possible to use more broadly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org