[jira] [Commented] (OOZIE-562) Design of Oozie Logging System

["Alejandro Abdelnur (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107809#comment-13107809 ] 

Alejandro Abdelnur commented on OOZIE-562:
------------------------------------------

By default Oozie logging level is DEBUG for most of its modules. Before putting Oozie in production, the log levels should be carefully revisited by the user. Wouldn't this be enough?

> Design of Oozie Logging System
> ------------------------------
>
>                 Key: OOZIE-562
>                 URL: https://issues.apache.org/jira/browse/OOZIE-562
>             Project: Oozie
>          Issue Type: Question
>            Reporter: Kiran Nagasubramanian
>
> When large log files are there in the log folder, even if we try to retrieve the log content(of size of the order of kBs) for a small job, it takes quite some significant amount of time, as equally as it takes to retrieve log content for very large jobs(of the order of GBs). This happens because, the list of files to be scanned for log retrieval is the same for all jobs that run approximately at around the same time.
> Chances that this might materialize in production systems is really high. Since, hundreds of jobs would be logging to the same file for an hr and this file size would be really huge. Is it possible to have the logs for the jobs separately so that scanning large log files of other jobs can be avoided? Would this be really worth the effort?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira