You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by fr...@apache.org on 2022/09/22 01:46:55 UTC
[druid] branch master updated: update log4j example (#13095)
This is an automated email from the ASF dual-hosted git repository.
frankchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new eb760c3d1d update log4j example (#13095)
eb760c3d1d is described below
commit eb760c3d1d314414ab8616c4a42796b20e798ed5
Author: Charles Smith <te...@gmail.com>
AuthorDate: Wed Sep 21 18:46:49 2022 -0700
update log4j example (#13095)
* update log4j example
* fix some style issues
* Update docs/configuration/logging.md
Co-authored-by: Frank Chen <fr...@apache.org>
Co-authored-by: Frank Chen <fr...@apache.org>
---
docs/configuration/logging.md | 123 +++++++++++++++++++++++++++---------------
1 file changed, 81 insertions(+), 42 deletions(-)
diff --git a/docs/configuration/logging.md b/docs/configuration/logging.md
index 1be881a0cb..c7b7242cb2 100644
--- a/docs/configuration/logging.md
+++ b/docs/configuration/logging.md
@@ -23,80 +23,118 @@ title: "Logging"
-->
-Apache Druid processes will emit logs that are useful for debugging to log files.
-These processes also emit periodic [metrics](../configuration/index.md#enabling-metrics) about their state.
-Metric info logs can be disabled with `-Ddruid.emitter.logging.logLevel=debug`.
+Apache Druid services emit logs that to help you debug.
+The same services also emit periodic [metrics](../configuration/index.md#enabling-metrics) about their state.
+To disable metric info logs set the following runtime property: `-Ddruid.emitter.logging.logLevel=debug`.
Druid uses [log4j2](http://logging.apache.org/log4j/2.x/) for logging.
-The default configuration file log4j2.xml ships with Druid under conf/druid/{config}/_common/log4j2.xml .
+The default configuration file log4j2.xml ships with Druid at the following path: `conf/druid/{config}/_common/log4j2.xml`.
By default, Druid uses `RollingRandomAccessFile` for rollover daily, and keeps log files up to 7 days.
-If that's not suitable in your case, you could modify the log4j2.xml to meet your need.
+If that's not suitable in your case, modify the `log4j2.xml` accordingly.
-An example log4j2.xml file is shown below:
+The following example log4j2.xml is based upon the micro quickstart:
```
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration status="WARN">
- <Console name="Console" target="SYSTEM_OUT">
- <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
- </Console>
-
- <RollingRandomAccessFile name="FileAppender"
- fileName="${sys:druid.log.path}/${sys:druid.node.type}.log"
- filePattern="${sys:druid.log.path}/${sys:druid.node.type}.%d{yyyyMMdd}.log">
- <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
- <Policies>
+ <Properties>
+ <!-- to change log directory, set DRUID_LOG_DIR environment variable to your directory before launching Druid -->
+ <Property name="druid.log.path" value="log" />
+ </Properties>
+
+ <Appenders>
+ <Console name="Console" target="SYSTEM_OUT">
+ <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
+ </Console>
+
+ <!-- Rolling Files-->
+ <RollingRandomAccessFile name="FileAppender"
+ fileName="${sys:druid.log.path}/${sys:druid.node.type}.log"
+ filePattern="${sys:druid.log.path}/${sys:druid.node.type}.%d{yyyyMMdd}.log">
+ <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
+ <Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
- </Policies>
- <DefaultRolloverStrategy>
- <Delete basePath="${sys:druid.log.path}/" maxDepth="1">
- <IfFileName glob="*.log" />
- <IfLastModified age="7d" />
- </Delete>
- </DefaultRolloverStrategy>
- </RollingRandomAccessFile>
+ </Policies>
+ <DefaultRolloverStrategy>
+ <Delete basePath="${sys:druid.log.path}/" maxDepth="1">
+ <IfFileName glob="*.log" />
+ <IfLastModified age="7d" />
+ </Delete>
+ </DefaultRolloverStrategy>
+ </RollingRandomAccessFile>
+
+ </Appenders>
+
<Loggers>
<Root level="info">
<AppenderRef ref="FileAppender"/>
</Root>
- <!-- Uncomment to enable logging of all HTTP requests
- <Logger name="org.apache.druid.jetty.RequestLog" additivity="false" level="DEBUG">
- <AppenderRef ref="FileAppender"/>
+ <!-- Set level="debug" to see stack traces for query errors -->
+ <Logger name="org.apache.druid.server.QueryResource" level="info" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+ <Logger name="org.apache.druid.server.QueryLifecycle" level="info" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+
+ <!-- Set level="debug" or "trace" to see more Coordinator details (segment balancing, load/drop rules, etc) -->
+ <Logger name="org.apache.druid.server.coordinator" level="info" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+
+ <!-- Set level="debug" to see low-level details about segments and ingestion -->
+ <Logger name="org.apache.druid.segment" level="info" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+
+ <!-- Set level="debug" to see more information about extension initialization -->
+ <Logger name="org.apache.druid.initialization" level="info" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+
+ <!-- Quieter logging at startup -->
+ <Logger name="org.skife.config" level="warn" additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
+ <Logger name="com.sun.jersey.guice" level="warn" additivity="false">
+ <Appender-ref ref="FileAppender"/>
</Logger>
- -->
</Loggers>
</Configuration>
```
> NOTE:
-> Although the log4j configuration file is shared with Druid's task peon processes,
-> the appenders in this file DO NOT take effect for peon processes, which always output logs to standard output.
+> Although Druid shares the log4j configuration file task peon processes,
+> the appenders in this file DO NOT take effect for peon processes. Peons always output logs to standard output.
> Middle Managers redirect task logs from standard output to [long-term storage](index.md#log-long-term-storage).
>
-> However, log level settings do take effect for these task peon processes,
-> which means you can still configure loggers at different logging level for task logs using `log4j2.xml`.
+> However, log level settings do take effect for these task peon processes.
+> This means you can configure loggers at different logging level for task logs using `log4j2.xml`.
## Log directory
-The included log4j2.xml configuration for Druid and ZooKeeper will output logs to the `log` directory at the root of the distribution.
+The included log4j2.xml configuration for Druid and ZooKeeper writes logs to the `log` directory at the root of the distribution.
If you want to change the log directory, set the environment variable `DRUID_LOG_DIR` to the right directory before you start Druid.
## All-in-one start commands
-If you use one of the all-in-one start commands, such as `bin/start-micro-quickstart`, then in the default configuration
-each service has two kind of log files. The main log file (for example, `log/historical.log`) is written by log4j2 and
-is rotated periodically.
+If you use one of the all-in-one start commands, such as `bin/start-micro-quickstart`, the default configuration for each service has two kinds of log files.
+Log4j2 writes the main log file and rotates it periodically.
+For example, `log/historical.log`.
-The secondary log file (for example, `log/historical.stdout.log`) contains anything that is written by the component
-directly to standard output or standard error without going through log4j2. This consists mainly of messages from the
-Java runtime itself. This file is not rotated, but it is generally small due to the low volume of messages. If
-necessary, you can truncate it using the Linux command `truncate --size 0 log/historical.stdout.log`.
+The secondary log file contains anything that is written by the component
+directly to standard output or standard error without going through log4j2.
+For example, `log/historical.stdout.log`.
+This consists mainly of messages from the
+Java runtime itself.
+This file is not rotated, but it is generally small due to the low volume of messages.
+If necessary, you can truncate it using the Linux command `truncate --size 0 log/historical.stdout.log`.
## Avoid reflective access warnings in logs
-On Java 11, you may see warnings like this in log files:
+On Java 11, you may see warnings like the following in the logs:
```
WARNING: An illegal reflective access operation has occurred
@@ -107,9 +145,10 @@ WARNING: All illegal access operations will be denied in a future release
To avoid these, add the `--add-exports` and `--add-opens` command line parameters described in the documentation section
about [Java strong encapsulation](../operations/java.md#strong-encapsulation).
-## My logs are really chatty, can I set them to asynchronously write?
+## Set the logs to asynchronously write
-Yes, using a `log4j2.xml` similar to the following causes some of the more chatty classes to write asynchronously:
+If your logs are really chatty, you can set them to write asynchronously.
+The following example shows a `log4j2.xml` that configures some of the more chatty classes to write asynchronously:
```
<?xml version="1.0" encoding="UTF-8" ?>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org