[jira] [Updated] (COMDEV-183) Add HTrace distributed tracing integration to YARN

["Colin Patrick McCabe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/COMDEV-183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Colin Patrick McCabe updated COMDEV-183:
----------------------------------------
    Summary: Add HTrace distributed tracing integration to YARN  (was: Add distributed tracing integration to YARN)

> Add HTrace distributed tracing integration to YARN
> --------------------------------------------------
>
>                 Key: COMDEV-183
>                 URL: https://issues.apache.org/jira/browse/COMDEV-183
>             Project: Community Development
>          Issue Type: New Feature
>            Reporter: Colin Patrick McCabe
>              Labels: Gsoc2016, gsoc, mentor
>
> Distributed tracing allows users to follow a request through the entire distributed system, crossing network and project boundaries.  The Apache HTrace project has added distributed tracing to HDFS and HBase (among other projects).  We should add tracing to YARN so that MapReduce, spark, and other frameworks can make use of it.
> Tracing should identify which yarn rpcs were made, and also tag top level trace spans with the yarn job that they are associated with.  That would let us analyze information relevant to a particular YARN job or set of jobs.  There are a lot of interesting projects here-- detecting hardware failures in production clusters, analyzing patterns in MR or Spark jobs, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)