[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

["Chris Trezzo (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Trezzo updated YARN-221:
------------------------------

    Attachment: YARN-221-trunk-v1.patch

Attached is a patch that provides the ability to configure log aggregation on a per container basis. All changes are at the yarn level. The main changes this patch makes are as follows:
1. Addition of a new set of methods to the ContainerLaunchContext that lets a yarn client set the log aggregation policy for that container.
2. A new set of log aggregation policies listed in the ContainerLogAggregationPolicy enum.
3. Modifications to the LogAggregationService and associated code paths to allow for per container configuration.
4. Addition of new unit tests and modification to existing tests to incorporate changes.

I am going to follow this patch with another patch in YARN-85 that will make the necessary changes at the MapReduce level to provide per job configuration of log aggregation.

Also, which repository should I list for yarn patches on reviews.apache.org? I see hdfs/common/mapreduce, but no yarn.

Feedback/+1's would be much appreciated. [~revans2] [~sandyr]

Thanks!

> NM should provide a way for AM to tell it not to aggregate logs.
> ----------------------------------------------------------------
>
>                 Key: YARN-221
>                 URL: https://issues.apache.org/jira/browse/YARN-221
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Robert Joseph Evans
>         Attachments: YARN-221-trunk-v1.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the logs should not be aggregated, that they should be aggregated with a high priority, or that they should be aggregated but with a lower priority.  The AM should be able to do this in the ContainerLaunch context to provide a default value, but should also be able to update the value when the container is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid connection to the NN at all.



--
This message was sent by Atlassian JIRA
(v6.1#6144)