[jira] [Commented] (SLIDER-810) YARN config changes to enable partial logs upload for long running services

["Gour Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363280#comment-14363280 ] 

Gour Saha commented on SLIDER-810:
----------------------------------

Comments from [~xgong]

Since YARN-3154 has been committed into trunk/branch-2/branch-2.7, I think that we need do some related changes on slider side about Log Aggregation for long running service (such as HBase).

We have added two more parameters in ApplicationSubmissionContext for log handling.

Originally, we have include_pattern/exclude_pattern to specified by the users to decide which log files will be uploaded while the application is running. This behavior is changed after we checked in the YARN-3154.

Right now, the meaning of old include_pattern/exclude_pattern has been changed. If the users specify the value for these two parameter. The log files specified for these two parameters will not be handled while the app is running, they will only be handled after the application finishes.

We introduced two new parameters: rolled_logs_include_pattern/rolled_logs_exclude_pattern in ApplicationSubmissionContext. If we need to upload logs while the app is running, we should use these two parameters. (They are just like the old include_pattern/exclude_pattern).

> YARN config changes to enable partial logs upload for long running services
> ---------------------------------------------------------------------------
>
>                 Key: SLIDER-810
>                 URL: https://issues.apache.org/jira/browse/SLIDER-810
>             Project: Slider
>          Issue Type: Bug
>            Reporter: Gour Saha
>
> As per bug YARN-3154, Slider needs to make appropriate changes to ApplicationSubmissionContext to enable partial logs upload.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)