You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Botong Huang (JIRA)" <ji...@apache.org> on 2016/09/23 21:16:20 UTC
[jira] [Comment Edited] (YARN-5525) Make log aggregation service
class configurable
[ https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517594#comment-15517594 ]
Botong Huang edited comment on YARN-5525 at 9/23/16 9:16 PM:
-------------------------------------------------------------
Thank [~subru] for the review and comments. Let me provide some context why we are doing this. By changing {{AppLogAggregator}} we can only customize the per app log aggregation behavior. One reason that we also want to customize {{LogAggregationService}} is to allow a different root log directory to upload logs to per app, whereas in the current implementation, logs in all apps are uploaded to a same (configurable) location in {{LogAggregationService.remoteRootLogDir}}. In general, if we want to make log aggregation pluggable, I think {{LogAggregationService}} is the right place because it is the entry service class.
Besides, I have a question regarding all the {{private}} to {{protected}} changes, where I am sure what is the right thing to do. In general, making log aggregation service class configurable will suffice and people can plug in any other implementation they need, without needing to modify {{LogAggregationService}} and {{AppLogAggregator}} at all. However, in our case, we only need to customize some features out of the current implementation. Rather copy all the code and modify on top, I implemented mine extending the current ones, so that lots of nice feature are inherited. The code is much less and easier to maintain, by simply override the methods that we need to customize. This is where the {{private}} to {{protected}} changes are needed so that the member variables and methods are visible to the subclasses.
was (Author: botong):
Thank [~subru] for the review and comments. Let me provide some context why we are doing this. By changing {{AppLogAggregator}} we can only customize the per app log aggregation behavior. One reason that we also want to customize {{LogAggregationService}} is to allow a different root log directory to upload logs to per app, whereas in the current implementation, logs in every app are uploaded to a same (configurable) location in {{LogAggregationService.remoteRootLogDir}}. In general, if we want to make log aggregation pluggable, I think {{LogAggregationService}} is the right place because it is the entry service class.
Besides, I have a question regarding all the {{private}} to {{protected}} changes, where I am sure what is the right thing to do. In general, making log aggregation service class configurable will suffice and people can plug in any other implementation they need, without needing to modify {{LogAggregationService}} and {{AppLogAggregator}} at all. However, in our case, we only need to customize some features out of the current implementation. Rather copy all the code and modify on top, I implemented mine extending the current ones, so that lots of nice feature are inherited. The code is much less and easier to maintain, by simply override the methods that we need to customize. This is where the {{private}} to {{protected}} changes are needed so that the member variables and methods are visible to the subclasses.
> Make log aggregation service class configurable
> -----------------------------------------------
>
> Key: YARN-5525
> URL: https://issues.apache.org/jira/browse/YARN-5525
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
> Reporter: Giovanni Matteo Fumarola
> Assignee: Botong Huang
> Priority: Minor
> Attachments: YARN-5525.v1.patch, YARN-5525.v2.patch, YARN-5525.v3.patch
>
>
> Make the log aggregation class configurable and extensible, so that alternative log aggregation behaviors like app specific log aggregation directory, log aggregation format can be implemented and plugged in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org