[jira] [Commented] (SAMZA-60) Specify log directory as a Java property for Log4j

["Chris Riccomini (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802035#comment-13802035 ] 

Chris Riccomini commented on SAMZA-60:
--------------------------------------

Poked around a bit more on this. Several notes:

1. It appears that SAMZA_LOG_DIR is only set for YARN jobs. For local jobs, the run-class.sh script just defaults to /tmp. This works for log directories, but for SAMZA_JOB_NAME, we'd need to set it for everything, since there's no default that makes sense.
2. Perhaps it makes sense to just set all config values as environment variables with the pattern SAMZA + Utils.escapeEnvironmentVariable(configKey.replace(".", "_"))?
3. If we don't go the route described in (2), it seems like the best integration point for SAMZA_JOB_NAME is in the CommandBuilder. We can add a setJobName(), and then set SAMZA_JOB_NAME in the buildEnvironment method in ShellCommandBuilder.

> Specify log directory as a Java property for Log4j
> --------------------------------------------------
>
>                 Key: SAMZA-60
>                 URL: https://issues.apache.org/jira/browse/SAMZA-60
>             Project: Samza
>          Issue Type: Bug
>          Components: container, yarn
>    Affects Versions: 0.6.0
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>
> We currently specify SAMZA_LOG_DIR as an environment variable when starting Samza's YARN AM, and also when starting a Samza container with YARN.
> It'd be nice to expose the Samza job's name as an environment variable, as well: SAMZA_JOB_NAME.
> It'd also be nice to use these variables in log4j.xml (for example), so we can have a file-based appender. Unfortunately, it appears that Log4j wants Java system properties, not environment variables.
> Let's update run-class.sh to append new Java system properties to SAMZA_OPTS:
> {noformat}
> SAMZA_OPTS="$SAMZA_OPTS -Dsamza.log.dir=$SAMZA_LOG_DIR -Dsamza.job.name=$SAMZA_JOB_NAME"
> {noformat}
> Then, in log4j.xml, we should be able to set a file appender:
> {noformat}
>       <appender  name = "samzaAppender" class="org.apache.log4j.RollingFileAppender">
>         ...
>         <param name="file" value="${samza.log.dir}/${samza.job.name}.log" />
>         ...
>     </appender>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)