You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by "Sumit Mohanty (JIRA)" <ji...@apache.org> on 2014/07/27 19:31:38 UTC
[jira] [Commented] (SLIDER-116) Log handling for long-lived
applications
[ https://issues.apache.org/jira/browse/SLIDER-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075656#comment-14075656 ]
Sumit Mohanty commented on SLIDER-116:
--------------------------------------
Modified the default log directories to be the root log folder. This is from container log folder:
{noformat}
./log/application_1405048900371_0031
./log/application_1405048900371_0031/container_1405048900371_0031_01_000001
./log/application_1405048900371_0031/container_1405048900371_0031_01_000001/slider-out.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000001/slider-err.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/status_command_stderr.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/output-1.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/status_command_stdout.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/errors-2.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/errors-1.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/output-2.txt
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/command-2.json
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/agent.log
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/command-1.json
./log/application_1405048900371_0031/container_1405048900371_0031_01_000002/status_command.json
{noformat}
This results in all log files to be in HDFS after application is stopped.
{noformat}
[yarn@c6403 bin]$ yarn logs --applicationId application_1405048900371_0031 | grep LogType
LogType: agent.log
LogType: command-1.json
LogType: command-2.json
LogType: errors-1.txt
LogType: errors-2.txt
LogType: output-1.txt
LogType: output-2.txt
LogType: status_command.json
LogType: status_command_stderr.txt
LogType: status_command_stdout.txt
LogType: slider-err.txt
LogType: slider-out.txt
{noformat}
Once YARN adds the capability to move logs to HDSF while application is alive Slider will be integrated with that capability.
> Log handling for long-lived applications
> ----------------------------------------
>
> Key: SLIDER-116
> URL: https://issues.apache.org/jira/browse/SLIDER-116
> Project: Slider
> Issue Type: Task
> Components: agent-provider
> Reporter: Sumit Mohanty
> Assignee: Sumit Mohanty
>
> Slider needs a solution for handling logs for long lived applications. The log files themselves could be huge and may or may not be ideal for pushing into HDFS. However, that should be an option. Otherwise, application owner should have an option of accessing logs from the local disk on hosts where container is/was active. Can YARN be configured to retain logs on a per application basis?
--
This message was sent by Atlassian JIRA
(v6.2#6252)