You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by ba...@apache.org on 2026/04/28 07:21:56 UTC
(apisix) branch master updated: docs: update http-logger, rocketmq-logger, and syslog plugin docs (#13272)
This is an automated email from the ASF dual-hosted git repository.
Baoyuantop pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apisix.git
The following commit(s) were added to refs/heads/master by this push:
new 52cba57a0 docs: update http-logger, rocketmq-logger, and syslog plugin docs (#13272)
52cba57a0 is described below
commit 52cba57a04230cf7cd015761eb80ed98427c75d1
Author: Yilia Lin <11...@users.noreply.github.com>
AuthorDate: Tue Apr 28 15:21:50 2026 +0800
docs: update http-logger, rocketmq-logger, and syslog plugin docs (#13272)
---
docs/en/latest/plugins/http-logger.md | 333 +++++++++++++-------
docs/en/latest/plugins/rocketmq-logger.md | 504 +++++++++++++++++++-----------
docs/en/latest/plugins/syslog.md | 317 ++++++++++++++-----
docs/zh/latest/plugins/http-logger.md | 341 +++++++++++++-------
docs/zh/latest/plugins/rocketmq-logger.md | 458 +++++++++++++++++++--------
docs/zh/latest/plugins/syslog.md | 326 ++++++++++++++-----
6 files changed, 1581 insertions(+), 698 deletions(-)
diff --git a/docs/en/latest/plugins/http-logger.md b/docs/en/latest/plugins/http-logger.md
index 5f93302bd..bbbcd8093 100644
--- a/docs/en/latest/plugins/http-logger.md
+++ b/docs/en/latest/plugins/http-logger.md
@@ -5,7 +5,7 @@ keywords:
- API Gateway
- Plugin
- HTTP Logger
-description: This document contains information about the Apache APISIX http-logger Plugin. Using this Plugin, you can push APISIX log data to HTTP or HTTPS servers.
+description: The http-logger Plugin pushes request and response logs as JSON objects to HTTP(S) servers in batches, allowing for customizable log formats to enhance data management.
---
<!--
@@ -27,28 +27,30 @@ description: This document contains information about the Apache APISIX http-log
#
-->
-## Description
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/http-logger" />
+</head>
-The `http-logger` Plugin is used to push log data requests to HTTP/HTTPS servers.
+## Description
-This will allow the ability to send log data requests as JSON objects to monitoring tools and other HTTP servers.
+The `http-logger` Plugin pushes request and response logs as JSON objects to HTTP(S) servers in batches and supports the customization of log formats.
## Attributes
-| Name | Type | Required | Default | Valid values | Description |
-| ---------------------- | ------- | -------- | ------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| uri | string | True | | | URI of the HTTP/HTTPS server. |
-| auth_header | string | False | | | Authorization headers if required. |
-| timeout | integer | False | 3 | [1,...] | Time to keep the connection alive for after sending a request. |
-| log_format | object | False | | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
-| include_req_body | boolean | False | false | [false, true] | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
-| include_req_body_expr | array | False | | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
-| max_req_body_bytes | integer | False | 524288 | >=1 | Request bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
-| include_resp_body | boolean | False | false | [false, true] | When set to `true` includes the response body in the log. |
-| include_resp_body_expr | array | False | | | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response if the expression evaluates to `true`. |
-| max_resp_body_bytes | integer | False | 524288 | >=1 | Response bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
-| concat_method | string | False | "json" | ["json", "new_line"] | Sets how to concatenate logs. When set to `json`, uses `json.encode` for all pending logs and when set to `new_line`, also uses `json.encode` but uses the newline (`\n`) to concatenate lines. |
-| ssl_verify | boolean | False | false | [false, true] | When set to `true` verifies the SSL certificate. |
+| Name | Type | Required | Default | Valid values | Description |
+|------------------------|---------|----------|---------|----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| uri | string | True | | | URI of the HTTP(S) server. |
+| auth_header | string | False | | | Authorization headers, if required by the HTTP(S) server. |
+| timeout | integer | False | 3 | greater than 0 | Time to keep the connection alive after sending a request. |
+| log_format | object | False | | | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). You can also configure log format on a global scale using the [plugin metadata](../terminology/plugin-metadata.md), which configures the log format for all `http-logger` Plugin instances. If the log format configured on the individual Plugin instance differ [...]
+| include_req_body | boolean | False | false | | If true, include the request body in the log. Note that if the request body is too big to be kept in the memory, it cannot be logged due to NGINX's limitations. |
+| include_req_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_req_body` is true. Request body is only logged when the expressions configured here evaluate to true. |
+| include_resp_body | boolean | False | false | | If true, include the response body in the log. |
+| include_resp_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_resp_body` is true. Response body is only logged when the expressions configured here evaluate to true. |
+| max_req_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum request body size in bytes to include in the log. If the request body exceeds this value, it will be truncated. |
+| max_resp_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum response body size in bytes to include in the log. If the response body exceeds this value, it will be truncated. |
+| concat_method | string | False | `json` | `json` or `new_line` | Method to concatenate logs. When set to `json`, use `json.encode` for all pending logs. When set to `new_line`, also use `json.encode` but use the newline character `\n` to concatenate lines. |
+| ssl_verify | boolean | False | false | | If true, verify the server's SSL certificate. |
:::note
@@ -56,58 +58,14 @@ This Plugin supports using batch processors to aggregate and process entries (lo
:::
-### Example of default log format
-
- ```json
- {
- "service_id": "",
- "apisix_latency": 100.99999809265,
- "start_time": 1703907485819,
- "latency": 101.99999809265,
- "upstream_latency": 1,
- "client_ip": "127.0.0.1",
- "route_id": "1",
- "server": {
- "version": "3.7.0",
- "hostname": "localhost"
- },
- "request": {
- "headers": {
- "host": "127.0.0.1:1984",
- "content-type": "application/x-www-form-urlencoded",
- "user-agent": "lua-resty-http/0.16.1 (Lua) ngx_lua/10025",
- "content-length": "12"
- },
- "method": "POST",
- "size": 194,
- "url": "http://127.0.0.1:1984/hello?log_body=no",
- "uri": "/hello?log_body=no",
- "querystring": {
- "log_body": "no"
- }
- },
- "response": {
- "headers": {
- "content-type": "text/plain",
- "connection": "close",
- "content-length": "12",
- "server": "APISIX/3.7.0"
- },
- "status": 200,
- "size": 123
- },
- "upstream": "127.0.0.1:1982"
- }
- ```
-
## Metadata
You can also set the format of the logs by configuring the Plugin metadata. The following configurations are available:
-| Name | Type | Required | Default | Description |
-| ---------- | ------ | -------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| log_format | object | False | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
-| max_pending_entries | integer | False | | Maximum number of pending entries that can be buffered in batch processor before it starts dropping them. |
+| Name | Type | Required | Description |
+|---------------------|---------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). |
+| max_pending_entries | integer | False | Maximum number of unprocessed entries allowed in the batch processor. When this limit is reached, new entries will be dropped until the backlog is reduced. Available in APISIX from version 3.15.0. |
:::info IMPORTANT
@@ -115,9 +73,14 @@ Configuring the Plugin metadata is global in scope. This means that it will take
:::
-The example below shows how you can configure through the Admin API:
+## Examples
+
+The examples below demonstrate how you can configure the `http-logger` Plugin for different scenarios.
+
+To follow along the examples, start a mock HTTP logging endpoint using [mockbin](https://mockbin.io) and note down the mockbin URL.
:::note
+
You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:
```bash
@@ -126,74 +89,220 @@ admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"/
:::
+### Log Requests in Default Log Format
+
+The following example demonstrates how you can configure the `http-logger` Plugin on a Route to log information of requests hitting the Route.
+
+Create a Route with the `http-logger` Plugin and configure the Plugin with your server URI:
+
```shell
-curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/http-logger \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr",
- "request": { "method": "$request_method", "uri": "$request_uri" },
- "response": { "status": "$status" }
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/"
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
```
-With this configuration, your logs would be formatted as shown below:
+Send a request to the Route:
```shell
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
+curl "http://127.0.0.1:9080/anything"
+```
+
+You should receive an `HTTP/1.1 200 OK` response. In your mockbin, you should see a log entry similar to the following:
+
+```json
+[
+ {
+ "upstream": "3.213.1.197:80",
+ "server": {
+ "hostname": "7d8d831179d4",
+ "version": "3.9.0"
+ },
+ "start_time": 1718291190508,
+ "client_ip": "192.168.65.1",
+ "response": {
+ "status": 200,
+ "headers": {
+ "server": "APISIX/3.9.0",
+ "content-length": "390",
+ "access-control-allow-credentials": "true",
+ "connection": "close",
+ "date": "Thu, 13 Jun 2024 15:06:31 GMT",
+ "access-control-allow-origin": "*",
+ "content-type": "application/json"
+ },
+ "size": 617
+ },
+ "latency": 1200.0000476837,
+ "upstream_latency": 1133,
+ "apisix_latency": 67.000047683716,
+ "request": {
+ "url": "http://127.0.0.1:9080/anything",
+ "querystring": {},
+ "method": "GET",
+ "uri": "/anything",
+ "headers": {
+ "accept": "*/*",
+ "user-agent": "curl/8.6.0",
+ "host": "127.0.0.1:9080"
+ },
+ "size": 85
+ },
+ "service_id": "",
+ "route_id": "http-logger-route"
+ }
+]
```
-## Enable Plugin
+### Log Request and Response Headers With Plugin Metadata
+
+The following example demonstrates how you can customize log format using [plugin metadata](../terminology/plugin-metadata.md) and NGINX variables to log specific headers from request and response.
+
+In APISIX, [plugin metadata](../terminology/plugin-metadata.md) is used to configure the common metadata fields of all Plugin instances of the same Plugin. It is useful when a Plugin is enabled across multiple resources and requires a universal update to their metadata fields.
-The example below shows how you can enable the Plugin on a specific Route:
+First, create a Route with the `http-logger` Plugin and configure the Plugin with your server URI:
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "plugins": {
- "http-logger": {
- "uri": "http://mockbin.org/bin/:ID"
- }
- },
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/"
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
},
- "uri": "/hello"
-}'
+ "type": "roundrobin"
+ }
+ }'
```
-As an example the [mockbin](http://mockbin.org/bin/create) server is used for mocking an HTTP server to see the logs produced by APISIX.
+Next, configure the Plugin metadata for `http-logger` to log the custom request header `env` and the response header `Content-Type`:
-## Example usage
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/http-logger" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "env": "$http_env",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
+```
-Now, if you make a request to APISIX, it will be logged in your mockbin server:
+Send a request to the Route with the `env` header:
```shell
-curl -i http://127.0.0.1:9080/hello
+curl "http://127.0.0.1:9080/anything" -H "env: dev"
+```
+
+You should receive an `HTTP/1.1 200 OK` response. In your mockbin, you should see a log entry similar to the following:
+
+```json
+[
+ {
+ "route_id": "http-logger-route",
+ "client_ip": "192.168.65.1",
+ "@timestamp": "2024-06-13T15:19:34+00:00",
+ "host": "127.0.0.1",
+ "env": "dev",
+ "resp_content_type": "application/json"
+ }
+]
```
-## Delete Plugin
+### Log Request Bodies Conditionally
-To disable this Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.
+The following example demonstrates how you can conditionally log request body.
+
+Create a Route with the `http-logger` Plugin as follows, enabling request body logging only when the URL query string `log_body` is `yes`:
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "uri": "/hello",
- "plugins": {},
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/",
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
+```
+
+Send a request to the Route with a URL query string satisfying the condition:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
```
+
+You should see the request body logged:
+
+```json
+[
+ {
+ "request": {
+ "url": "http://127.0.0.1:9080/anything?log_body=yes",
+ "querystring": {
+ "log_body": "yes"
+ },
+ "uri": "/anything?log_body=yes",
+ "body": "{\"env\": \"dev\"}"
+ }
+ }
+]
+```
+
+Send a request to the Route without any URL query string:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+You should not observe the request body in the log.
+
+:::note
+
+If you have customized the `log_format` in addition to setting `include_req_body` or `include_resp_body` to `true`, the Plugin would not include the bodies in the logs.
+
+As a workaround, you may be able to use the NGINX variable `$request_body` in the log format, such as:
+
+```json
+{
+ "http-logger": {
+ "log_format": {"body": "$request_body"}
+ }
+}
+```
+
+:::
diff --git a/docs/en/latest/plugins/rocketmq-logger.md b/docs/en/latest/plugins/rocketmq-logger.md
index 7da26a780..96a6db7b4 100644
--- a/docs/en/latest/plugins/rocketmq-logger.md
+++ b/docs/en/latest/plugins/rocketmq-logger.md
@@ -5,7 +5,7 @@ keywords:
- API Gateway
- Plugin
- RocketMQ Logger
-description: This document contains information about the Apache APISIX rocketmq-logger Plugin.
+description: The rocketmq-logger Plugin pushes request and response logs as JSON objects to RocketMQ clusters in batches, allowing for customizable log formats to enhance data management.
---
<!--
#
@@ -26,260 +26,384 @@ description: This document contains information about the Apache APISIX rocketmq
#
-->
-## Description
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/rocketmq-logger" />
+</head>
-The `rocketmq-logger` Plugin provides the ability to push logs as JSON objects to your RocketMQ clusters.
+## Description
-It might take some time to receive the log data. It will be automatically sent after the timer function in the [batch processor](../batch-processor.md) expires.
+The `rocketmq-logger` Plugin pushes request and response logs as JSON objects to RocketMQ clusters in batches and supports the customization of log formats.
## Attributes
-| Name | Type | Required | Default | Valid values | Description |
-|------------------------|---------|----------|-------------------------------------------------------------------------------|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| nameserver_list | object | True | | | List of RocketMQ nameservers. |
-| topic | string | True | | | Target topic to push the data to. |
-| key | string | False | | | Key of the messages. |
-| tag | string | False | | | Tag of the messages. |
-| log_format | object | False | | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
-| timeout | integer | False | 3 | [1,...] | Timeout for the upstream to send data. |
-| use_tls | boolean | False | false | | When set to `true`, uses TLS. |
-| access_key | string | False | "" | | Access key for ACL. Setting to an empty string will disable the ACL. |
-| secret_key | string | False | "" | | secret key for ACL. |
-| name | string | False | "rocketmq logger" | | Unique identifier for the batch processor. If you use Prometheus to monitor APISIX metrics, the name is exported in `apisix_batch_process_entries`. processor. |
-| meta_format | enum | False | "default" | ["default"，"origin"] | Format to collect the request information. Setting to `default` collects the information in JSON format and `origin` collects the information with the original HTTP request. See [examples](#meta_format-example) below. |
-| include_req_body | boolean | False | false | [false, true] | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
-| include_req_body_expr | array | False | | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
-| max_req_body_bytes | integer | False | 524288 | >=1 | Request bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
-| include_resp_body | boolean | False | false | [false, true] | When set to `true` includes the response body in the log. |
-| include_resp_body_expr | array | False | | | Filter for when the `include_resp_body` attribute is set to `true`. Response body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
-| max_resp_body_bytes | integer | False | 524288 | >=1 | Response bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
+| Name | Type | Required | Default | Valid values | Description |
+|------------------------|---------|----------|--------------------|-----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| nameserver_list | array[string] | True | | | List of RocketMQ nameservers. |
+| topic | string | True | | | Target topic to push the data to. |
+| key | string | False | | | Key of the message. |
+| tag | string | False | | | Tag of the message. |
+| log_format | object | False | | | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). You can also configure log format on a global scale using the [plugin metadata](../terminology/plugin-metadata.md), which configures the log format for all `rocketmq-logger` Plugin instances. If the log format configured on the individual Plugin [...]
+| timeout | integer | False | 3 | | Timeout for the upstream to send data. |
+| use_tls | boolean | False | false | | If true, enable TLS encryption for the connection. |
+| access_key | string | False | | | Access key for ACL. Setting to an empty string will disable the ACL. |
+| secret_key | string | False | | | Secret key for ACL. |
+| name | string | False | `rocketmq logger` | | Unique identifier of the Plugin for the batch processor. If you use Prometheus to monitor APISIX metrics, the name is exported in `apisix_batch_process_entries`. |
+| meta_format | string | False | `default` | `default` or `origin` | Format to collect the request information. Setting to `default` collects the information in JSON format and `origin` collects the information with the original HTTP request. |
+| include_req_body | boolean | False | false | | If true, include the request body in the log. Note that if the request body is too big to be kept in the memory, it cannot be logged due to NGINX's limitations. |
+| include_req_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_req_body` is true. Request body is only logged when the expressions configured here evaluate to true. |
+| include_resp_body | boolean | False | false | | If true, include the response body in the log. |
+| include_resp_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_resp_body` is true. Response body is only logged when the expressions configured here evaluate to true. |
+| max_req_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum request body size in bytes to include in the log. If the request body exceeds this value, it will be truncated. Available in APISIX from 3.16.0. |
+| max_resp_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum response body size in bytes to include in the log. If the response body exceeds this value, it will be truncated. Available in APISIX from 3.16.0. |
+| batch_max_size | integer | False | 1000 | greater than 0 | The number of log entries allowed in one batch. Once reached, the batch will be sent to the logging service. Setting this parameter to 1 means immediate processing. |
+| inactive_timeout | integer | False | 5 | greater than 0 | The maximum time in seconds to wait for new logs before sending the batch to the logging service. The value should be smaller than `buffer_duration`. |
+| buffer_duration | integer | False | 60 | greater than 0 | The maximum time in seconds from the earliest entry allowed before sending the batch to the logging service. |
+| retry_delay | integer | False | 1 | greater than or equal to 0 | The time interval in seconds to retry sending the batch to the logging service if the batch was not successfully sent. |
+| max_retry_count | integer | False | 0 | greater than or equal to 0 | The maximum number of unsuccessful retries allowed before dropping the log entries. |
NOTE: `encrypt_fields = {"secret_key"}` is also defined in the schema, which means that the field will be stored encrypted in etcd. See [encrypted storage fields](../plugin-develop.md#encrypted-storage-fields).
-This Plugin supports using batch processors to aggregate and process entries (logs/data) in a batch. This avoids the need for frequently submitting the data. The batch processor submits data every `5` seconds or when the data in the queue reaches `1000`. See [Batch Processor](../batch-processor.md#configuration) for more information or setting your custom configuration.
+## Metadata
-:::info IMPORTANT
+You can also set the format of the logs by configuring the Plugin metadata. The following configurations are available:
-The data is first written to a buffer. When the buffer exceeds the `batch_max_size` or `buffer_duration` attribute, the data is sent to the RocketMQ server and the buffer is flushed.
+| Name | Type | Required | Description |
+|---------------------|---------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). |
+| max_pending_entries | integer | False | Maximum number of unprocessed entries allowed in the batch processor. When this limit is reached, new entries will be dropped until the backlog is reduced. Available in APISIX from version 3.15.0. |
-If the process is successful, it will return `true` and if it fails, returns `nil` with a string with the "buffer overflow" error.
+:::info IMPORTANT
+
+Configuring the Plugin metadata is global in scope. This means that it will take effect on all Routes and Services which use the `rocketmq-logger` Plugin.
:::
-### meta_format example
+## Examples
+
+The examples below demonstrate how you can configure the `rocketmq-logger` Plugin for different scenarios.
+
+To follow along the examples, start a sample RocketMQ cluster:
+
+```yaml title="docker-compose.yml"
+version: "3"
+
+services:
+ rocketmq_namesrv:
+ image: apacherocketmq/rocketmq:4.6.0
+ container_name: rmqnamesrv
+ restart: unless-stopped
+ ports:
+ - "9876:9876"
+ command: sh mqnamesrv
+ networks:
+ rocketmq_net:
+
+ rocketmq_broker:
+ image: apacherocketmq/rocketmq:4.6.0
+ container_name: rmqbroker
+ restart: unless-stopped
+ ports:
+ - "10909:10909"
+ - "10911:10911"
+ - "10912:10912"
+ depends_on:
+ - rocketmq_namesrv
+ command: sh mqbroker -n rmqnamesrv:9876 -c ../conf/broker.conf
+ networks:
+ rocketmq_net:
+
+networks:
+ rocketmq_net:
+```
-- default:
+Start containers:
-```json
- {
- "upstream": "127.0.0.1:1980",
- "start_time": 1619414294760,
- "client_ip": "127.0.0.1",
- "service_id": "",
- "route_id": "1",
- "request": {
- "querystring": {
- "ab": "cd"
- },
- "size": 90,
- "uri": "/hello?ab=cd",
- "url": "http://localhost:1984/hello?ab=cd",
- "headers": {
- "host": "localhost",
- "content-length": "6",
- "connection": "close"
- },
- "method": "GET"
- },
- "response": {
- "headers": {
- "connection": "close",
- "content-type": "text/plain; charset=utf-8",
- "date": "Mon, 26 Apr 2021 05:18:14 GMT",
- "server": "APISIX/2.5",
- "transfer-encoding": "chunked"
- },
- "size": 190,
- "status": 200
- },
- "server": {
- "hostname": "localhost",
- "version": "2.5"
- },
- "latency": 0
- }
+```shell
+docker compose up -d
```
-- origin:
+In a few seconds, the name server and broker should start.
-```http
- GET /hello?ab=cd HTTP/1.1
- host: localhost
- content-length: 6
- connection: close
+Create the `TopicTest` topic:
- abcdef
+```shell
+docker exec -i rmqnamesrv rm /home/rocketmq/rocketmq-4.6.0/conf/tools.yml
+docker exec -i rmqnamesrv /home/rocketmq/rocketmq-4.6.0/bin/mqadmin updateTopic -n rmqnamesrv:9876 -t TopicTest -c DefaultCluster
```
-### meta_format example
-
-- `default`:
-
- ```json
- {
- "upstream": "127.0.0.1:1980",
- "start_time": 1619414294760,
- "client_ip": "127.0.0.1",
- "service_id": "",
- "route_id": "1",
- "request": {
- "querystring": {
- "ab": "cd"
- },
- "size": 90,
- "uri": "/hello?ab=cd",
- "url": "http://localhost:1984/hello?ab=cd",
- "headers": {
- "host": "localhost",
- "content-length": "6",
- "connection": "close"
- },
- "body": "abcdef",
- "method": "GET"
- },
- "response": {
- "headers": {
- "connection": "close",
- "content-type": "text/plain; charset=utf-8",
- "date": "Mon, 26 Apr 2021 05:18:14 GMT",
- "server": "APISIX/2.5",
- "transfer-encoding": "chunked"
- },
- "size": 190,
- "status": 200
- },
- "server": {
- "hostname": "localhost",
- "version": "2.5"
- },
- "latency": 0
- }
- ```
-
-- `origin`:
+Wait for messages in the configured RocketMQ topic:
- ```http
- GET /hello?ab=cd HTTP/1.1
- host: localhost
- content-length: 6
- connection: close
+```shell
+docker run -it --name rocketmq_consumer -e NAMESRV_ADDR=localhost:9876 --net host apacherocketmq/rocketmq:4.6.0 sh tools.sh org.apache.rocketmq.example.quickstart.Consumer
+```
- abcdef
- ```
+In a few seconds, the consumer should start and listen for messages from APISIX:
-## Metadata
+```text
+Consumer Started.
+```
-You can also set the format of the logs by configuring the Plugin metadata. The following configurations are available:
+Open a new terminal session for the following steps working with APISIX.
-| Name | Type | Required | Default | Description |
-|------------|--------|----------|-------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| log_format | object | False | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
-| max_pending_entries | integer | False | | Maximum number of pending entries that can be buffered in batch processor before it starts dropping them. |
+:::note
-:::info IMPORTANT
+You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:
-Configuring the Plugin metadata is global in scope. This means that it will take effect on all Routes and Services which use the `rocketmq-logger` Plugin.
+```bash
+admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
+```
:::
-The example below shows how you can configure through the Admin API:
+### Log in Different Meta Log Formats
-:::note
-You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:
+The following example demonstrates how you can enable the `rocketmq-logger` Plugin on a Route, which logs client requests to the Route and pushes logs to RocketMQ. You will also understand the differences between the `default` and `origin` meta log formats.
-```bash
-admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
+Create a Route with `rocketmq-logger` with `meta_format` set to the `default` log format and `batch_max_size` set to 1 to send the log entry immediately:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
-:::
+Send a request to the Route to generate a log entry:
```shell
-curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/rocketmq-logger -H "X-API-KEY: $admin_key" -X PUT -d '
+curl -i "http://127.0.0.1:9080/anything"
+```
+
+You should see a log entry similar to the following:
+
+```json
{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr",
- "request": { "method": "$request_method", "uri": "$request_uri" },
- "response": { "status": "$status" }
+ "client_ip": "127.0.0.1",
+ "upstream": "34.197.122.172:80",
+ "start_time": 1744727400000,
+ "request": {
+ "headers": {
+ "host": "127.0.0.1:9080",
+ "accept": "*/*",
+ "user-agent": "curl/8.6.0"
+ },
+ "querystring": {},
+ "size": 86,
+ "uri": "/anything",
+ "url": "http://127.0.0.1:9080/anything",
+ "method": "GET"
+ },
+ "route_id": "rocketmq-logger-route",
+ "apisix_latency": 8.9998455047607,
+ "upstream_latency": 503,
+ "latency": 511.99984550476,
+ "response": {
+ "size": 617,
+ "headers": {
+ "content-length": "391",
+ "connection": "close",
+ "date": "Tue, 15 Apr 2025 14:30:00 GMT",
+ "server": "APISIX/3.15.0",
+ "content-type": "application/json"
+ },
+ "status": 200
+ },
+ "server": {
+ "hostname": "apisix",
+ "version": "3.15.0"
+ },
+ "service_id": ""
+}
+```
+
+Update the `rocketmq-logger` meta log format to `origin`:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/rocketmq-logger-route" -X PATCH \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "plugins": {
+ "rocketmq-logger": {
+ "meta_format": "origin"
+ }
}
-}'
+ }'
```
-With this configuration, your logs would be formatted as shown below:
+Send a request to the Route again to generate a new log entry:
```shell
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
+curl -i "http://127.0.0.1:9080/anything"
```
-## Enable Plugin
+You should see a log entry in the raw HTTP request format:
+
+```text
+GET /anything HTTP/1.1
+host: 127.0.0.1:9080
+user-agent: curl/8.6.0
+accept: */*
+```
+
+### Log Request and Response Headers With Plugin Metadata
+
+The following example demonstrates how you can customize log format using [plugin metadata](../terminology/plugin-metadata.md) and NGINX variables to log specific headers from request and response.
+
+In APISIX, [plugin metadata](../terminology/plugin-metadata.md) is used to configure the common metadata fields of all Plugin instances of the same Plugin. It is useful when a Plugin is enabled across multiple resources and requires a universal update to their metadata fields.
-The example below shows how you can enable the `rocketmq-logger` Plugin on a specific Route:
+Note that customizing log format with Plugin metadata requires `meta_format` to be set to `default`. If `meta_format` is set to `origin`, the log entries will remain in `origin` format.
+
+First, create a Route with `rocketmq-logger` with `meta_format` set to `default` and `batch_max_size` set to 1 to send the log entry immediately:
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/5 -H "X-API-KEY: $admin_key" -X PUT -d '
-{
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
"plugins": {
- "rocketmq-logger": {
- "nameserver_list" : [ "127.0.0.1:9876" ],
- "topic" : "test2",
- "batch_max_size": 1,
- "name": "rocketmq logger"
- }
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1
+ }
},
"upstream": {
- "nodes": {
- "127.0.0.1:1980": 1
- },
- "type": "roundrobin"
- },
- "uri": "/hello"
-}'
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
-This Plugin also supports pushing to more than one nameserver at a time. You can specify multiple nameserver in the Plugin configuration as shown below:
+Next, configure the Plugin metadata for `rocketmq-logger` to log the custom request header `env` and the response header `Content-Type`:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/rocketmq-logger" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "env": "$http_env",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
+```
+
+Send a request to the Route with the `env` header:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -H "env: dev"
+```
+
+You should see a log entry similar to the following:
```json
-"nameserver_list" : [
- "127.0.0.1:9876",
- "127.0.0.2:9876"
-]
+{
+ "host": "127.0.0.1",
+ "client_ip": "127.0.0.1",
+ "resp_content_type": "application/json",
+ "route_id": "rocketmq-logger-route",
+ "env": "dev",
+ "@timestamp": "2025-04-15T14:30:00+00:00"
+}
```
-## Example usage
+### Log Request Bodies Conditionally
+
+The following example demonstrates how you can conditionally log request body.
-Now, if you make a request to APISIX, it will be logged in your RocketMQ server:
+Create a Route with `rocketmq-logger` as follows, enabling request body logging only when the URL query string `log_body` is `yes`:
```shell
-curl -i http://127.0.0.1:9080/hello
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1,
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
+```
+
+Send a request to the Route with a URL query string satisfying the condition:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
```
-## Delete Plugin
+You should see the request body logged:
+
+```json
+{
+ "request": {
+ "method": "POST",
+ "body": "{\"env\": \"dev\"}",
+ "size": 183
+ }
+}
+```
-To remove the `rocketmq-logger` Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload, and you do not have to restart for this to take effect.
+Send a request to the Route without any URL query string:
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d '
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+You should not observe the request body in the log.
+
+:::note
+
+If you have customized the `log_format` in addition to setting `include_req_body` or `include_resp_body` to `true`, the Plugin would not include the bodies in the logs.
+
+As a workaround, you may be able to use the NGINX variable `$request_body` in the log format, such as:
+
+```json
{
- "methods": ["GET"],
- "uri": "/hello",
- "plugins": {},
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
- }
-}'
+ "rocketmq-logger": {
+ "log_format": {"body": "$request_body"}
+ }
+}
```
+
+:::
diff --git a/docs/en/latest/plugins/syslog.md b/docs/en/latest/plugins/syslog.md
index 9966dac76..13eb01690 100644
--- a/docs/en/latest/plugins/syslog.md
+++ b/docs/en/latest/plugins/syslog.md
@@ -5,7 +5,7 @@ keywords:
- API Gateway
- Plugin
- Syslog
-description: This document contains information about the Apache APISIX syslog Plugin.
+description: The syslog Plugin pushes request and response logs as JSON objects to syslog servers in batches, allowing for customizable log formats to enhance data management.
---
<!--
#
@@ -26,48 +26,53 @@ description: This document contains information about the Apache APISIX syslog P
#
-->
-## Description
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/syslog" />
+</head>
-The `syslog` Plugin is used to push logs to a Syslog server.
+## Description
-Logs can be set as JSON objects.
+The `syslog` Plugin pushes request and response logs as JSON objects to syslog servers in batches and supports the customization of log formats.
## Attributes
-| Name | Type | Required | Default | Valid values | Description |
-|------------------|---------|----------|--------------|---------------|--------------------------------------------------------------------------------------------------------------------------|
-| host | string | True | | | IP address or the hostname of the Syslog server. |
-| port | integer | True | | | Target port of the Syslog server. |
-| name | string | False | "sys logger" | | Identifier for the server. If you use Prometheus to monitor APISIX metrics, the name is exported in `apisix_batch_process_entries`. |
-| timeout | integer | False | 3000 | [1, ...] | Timeout in ms for the upstream to send data. |
-| tls | boolean | False | false | | When set to `true` performs TLS verification. |
-| flush_limit | integer | False | 4096 | [1, ...] | Maximum size of the buffer (KB) and the current message before it is flushed and written to the server. |
-| drop_limit | integer | False | 1048576 | | Maximum size of the buffer (KB) and the current message before the current message is dropped because of the size limit. |
-| sock_type | string | False | "tcp" | ["tcp", "udp] | Transport layer protocol to use. |
-| pool_size | integer | False | 5 | [5, ...] | Keep-alive pool size used by `sock:keepalive`. |
-| log_format | object | False | | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
-| include_req_body | boolean | False | false | [false, true] | When set to `true` includes the request body in the log. |
-| include_req_body_expr | array | False | | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
-| max_req_body_bytes | integer | False | 524288 | >=1 | Request bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
-| include_resp_body | boolean | False | false | [false, true] | When set to `true` includes the response body in the log. |
-| include_resp_body_expr | array | False | | | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response if the expression evaluates to `true`. |
-| max_resp_body_bytes | integer | False | 524288 | >=1 | Response bodies within this size will be logged, if the size exceeds the configured value it will be truncated before logging. |
+| Name | Type | Required | Default | Valid values | Description |
+|------------------------|---------|----------|--------------|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| host | string | True | | | IP address or hostname of the syslog server. |
+| port | integer | True | | | Target port of the syslog server. |
+| timeout | integer | False | 3000 | greater than 0 | Timeout for the upstream to send data, in milliseconds. |
+| tls | boolean | False | false | | If true, verify TLS. |
+| flush_limit | integer | False | 4096 | greater than 0 | Maximum size of the buffer and the current message in bytes (B) before the logs are pushed to the syslog server. |
+| drop_limit | integer | False | 1048576 | greater than 0 | Maximum size of the buffer and the current message allowed in bytes (B) before the logs are dropped. |
+| sock_type | string | False | `tcp` | `tcp` or `udp` | Transport layer protocol to use. |
+| pool_size | integer | False | 5 | greater than or equal to 5 | Keep-alive pool size used by `sock:keepalive`. |
+| log_format | object | False | | | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). You can also configure log format on a global scale using the [plugin metadata](../terminology/plugin-metadata.md), which configures the log format for all `syslog` Plugin instances. If the log format configured on the individual Plugin instance diffe [...]
+| include_req_body | boolean | False | false | | If true, include the request body in the log. Note that if the request body is too big to be kept in the memory, it cannot be logged due to NGINX's limitations. |
+| include_req_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_req_body` is true. Request body is only logged when the expressions configured here evaluate to true. |
+| include_resp_body | boolean | False | false | | If true, include the response body in the log. |
+| include_resp_body_expr | array | False | | | An array of one or more conditions in the form of [lua-resty-expr](https://github.com/api7/lua-resty-expr) expressions. Used when `include_resp_body` is true. Response body is only logged when the expressions configured here evaluate to true. |
+| max_req_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum request body size in bytes to include in the log. If the request body exceeds this value, it will be truncated. Available in APISIX from 3.16.0. |
+| max_resp_body_bytes | integer | False | 524288 | greater than or equal to 1 | Maximum response body size in bytes to include in the log. If the response body exceeds this value, it will be truncated. Available in APISIX from 3.16.0. |
+| name | string | False | `sys logger` | | Unique identifier of the Plugin for the batch processor. If you use Prometheus to monitor APISIX metrics, the name is exported in `apisix_batch_process_entries`. |
+| batch_max_size | integer | False | 1000 | greater than 0 | The number of log entries allowed in one batch. Once reached, the batch will be sent to the logging service. Setting this parameter to 1 means immediate processing. |
+| inactive_timeout | integer | False | 5 | greater than 0 | The maximum time in seconds to wait for new logs before sending the batch to the logging service. The value should be smaller than `buffer_duration`. |
+| buffer_duration | integer | False | 60 | greater than 0 | The maximum time in seconds from the earliest entry allowed before sending the batch to the logging service. |
+| retry_delay | integer | False | 1 | greater than or equal to 0 | The time interval in seconds to retry sending the batch to the logging service if the batch was not successfully sent. |
+| max_retry_count | integer | False | 0 | greater than or equal to 0 | The maximum number of unsuccessful retries allowed before dropping the log entries. |
-This Plugin supports using batch processors to aggregate and process entries (logs/data) in a batch. This avoids the need for frequently submitting the data. The batch processor submits data every `5` seconds or when the data in the queue reaches `1000`. See [Batch Processor](../batch-processor.md#configuration) for more information or setting your custom configuration.
+:::note
-### meta_format example
+This Plugin supports using batch processors to aggregate and process entries (logs/data) in a batch. This avoids the need for frequently submitting the data. The batch processor submits data every `5` seconds or when the data in the queue reaches `1000`. See [Batch Processor](../batch-processor.md#configuration) for more information or setting your custom configuration.
-```text
-"<46>1 2024-01-06T02:30:59.145Z 127.0.0.1 apisix 82324 - - {\"response\":{\"status\":200,\"size\":141,\"headers\":{\"content-type\":\"text/plain\",\"server\":\"APISIX/3.7.0\",\"transfer-encoding\":\"chunked\",\"connection\":\"close\"}},\"route_id\":\"1\",\"server\":{\"hostname\":\"baiyundeMacBook-Pro.local\",\"version\":\"3.7.0\"},\"request\":{\"uri\":\"/opentracing\",\"url\":\"http://127.0.0.1:1984/opentracing\",\"querystring\":{},\"method\":\"GET\",\"size\":155,\"headers\":{\"content-t [...]
-```
+:::
## Metadata
You can also set the format of the logs by configuring the Plugin metadata. The following configurations are available:
-| Name | Type | Required | Default | Description |
-| ---------- | ------ | -------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| log_format | object | False | | Log format declared as key-value pairs in JSON. Values support strings and nested objects (up to five levels deep; deeper fields are truncated). Within strings, [APISIX](../apisix-variable.md) or [NGINX](http://nginx.org/en/docs/varindex.html) variables can be referenced by prefixing with `$`. |
+| Name | Type | Required | Description |
+|------------|--------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | Custom log format using key-value pairs in JSON format. Values can reference [NGINX variables](https://nginx.org/en/docs/http/ngx_http_core_module.html). |
:::info IMPORTANT
@@ -75,9 +80,18 @@ Configuring the Plugin metadata is global in scope. This means that it will take
:::
-The example below shows how you can configure through the Admin API:
+## Examples
+
+The examples below demonstrate how you can configure the `syslog` Plugin for different scenarios.
+
+To follow along the examples, start an example rsyslog server in Docker:
+
+```shell
+docker run -d -p 514:514 --name example-rsyslog-server rsyslog/syslog_appliance_alpine
+```
:::note
+
You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:
```bash
@@ -86,73 +100,228 @@ admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"/
:::
+### Push Log to Syslog Server
+
+The following example demonstrates how you can enable the `syslog` Plugin on a Route, which logs client requests to the Route and pushes logs to the syslog server.
+
+Create a Route with `syslog`, replacing the `host` and `port` with the address and port of your syslog server. Set `flush_limit` to 1 to push log to the syslog server immediately:
+
```shell
-curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/syslog -H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr",
- "request": { "method": "$request_method", "uri": "$request_uri" },
- "response": { "status": "$status" }
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
+ "plugins": {
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
```
-With this configuration, your logs would be formatted as shown below:
+Send a request to the Route:
```shell
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
+curl -i "http://127.0.0.1:9080/anything"
```
-## Enable Plugin
+You should receive an `HTTP/1.1 200 OK` response.
-The example below shows how you can enable the Plugin for a specific Route:
+In the syslog server, you should see a log entry similar to the following:
-```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d '
+```json
{
+ "response": {
+ "status": 200,
+ "headers": {
+ "access-control-allow-credentials": "true",
+ "connection": "close",
+ "date": "Sat, 02 Mar 2024 00:14:19 GMT",
+ "access-control-allow-origin": "*",
+ "server": "APISIX/3.8.0",
+ "content-type": "application/json",
+ "content-length": "387"
+ },
+ "size": 614
+ },
+ "service_id": "",
+ "client_ip": "172.19.0.1",
+ "server": {
+ "hostname": "eff61bf7be4d",
+ "version": "3.8.0"
+ },
+ "upstream": "35.171.123.176:80",
+ "apisix_latency": 13.999900817871,
+ "request": {
+ "method": "GET",
+ "url": "http://127.0.0.1:9080/anything",
+ "querystring": {},
+ "size": 86,
+ "uri": "/anything",
+ "headers": {
+ "host": "127.0.0.1:9080",
+ "accept": "*/*",
+ "user-agent": "curl/7.29.0"
+ }
+ },
+ "route_id": "syslog-route",
+ "upstream_latency": 165,
+ "latency": 178.99990081787,
+ "start_time": 1709334859598
+}
+```
+
+### Customize Log Format With Plugin Metadata
+
+The following example demonstrates how you can customize log format using [plugin metadata](../terminology/plugin-metadata.md). The log format configured in Plugin metadata will apply to all `syslog` Plugin instances.
+
+Create a Route with the `syslog` Plugin:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
"plugins": {
- "syslog": {
- "host" : "127.0.0.1",
- "port" : 5044,
- "flush_limit" : 1
- }
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1
+ }
},
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
- },
- "uri": "/hello"
-}'
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
-## Example usage
+Configure Plugin metadata for `syslog`:
-Now, if you make a request to APISIX, it will be logged in your Syslog server:
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/syslog" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "route_id": "$route_id",
+ "client_ip": "$remote_addr",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
+```
+
+Send a request to the Route:
```shell
-curl -i http://127.0.0.1:9080/hello
+curl -i "http://127.0.0.1:9080/anything"
```
-## Delete Plugin
+In the syslog server, you should see a log entry similar to the following:
+
+```json
+{
+ "@timestamp": "2024-03-02T00:00:31+00:00",
+ "resp_content_type": "application/json",
+ "host": "127.0.0.1",
+ "route_id": "syslog-route",
+ "client_ip": "172.19.0.1"
+}
+```
-To remove the `syslog` Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.
+### Log Request Bodies Conditionally
+
+The following example demonstrates how you can conditionally log request body.
+
+Create a Route with the `syslog` Plugin, enabling request body logging only when the URL query string `log_body` is `yes`:
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "methods": ["GET"],
- "uri": "/hello",
- "plugins": {},
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
+ "plugins": {
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1,
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
+```
+
+Send a request to the Route with a URL query string satisfying the condition:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
+```
+
+You should see the request body logged:
+
+```json
+{
+ "request": {
+ "method": "POST",
+ "url": "http://127.0.0.1:9080/anything?log_body=yes",
+ "querystring": {
+ "log_body": "yes"
+ },
+ "size": 183,
+ "body": "{\"env\": \"dev\"}",
+ "uri": "/anything?log_body=yes",
+ "headers": {
+ "accept": "*/*",
+ "user-agent": "curl/7.29.0",
+ "host": "127.0.0.1:9080",
+ "content-type": "application/x-www-form-urlencoded",
+ "content-length": "14"
}
-}'
+ }
+}
```
+
+Send a request to the Route without any URL query string:
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+You should not observe the request body in the log.
+
+:::note
+
+If you have customized the `log_format` in addition to setting `include_req_body` or `include_resp_body` to `true`, the Plugin would not include the bodies in the logs.
+
+As a workaround, you may be able to use the NGINX variable `$request_body` in the log format, such as:
+
+```json
+{
+ "syslog": {
+ "log_format": {"body": "$request_body"}
+ }
+}
+```
+
+:::
diff --git a/docs/zh/latest/plugins/http-logger.md b/docs/zh/latest/plugins/http-logger.md
index 6c45bc5e8..a73caaa99 100644
--- a/docs/zh/latest/plugins/http-logger.md
+++ b/docs/zh/latest/plugins/http-logger.md
@@ -6,10 +6,9 @@ keywords:
- 插件
- HTTP Logger
- 日志
-description: 本文介绍了 API 网关 Apache APISIX 的 http-logger 插件。使用该插件可以将 APISIX 的日志数据推送到 HTTP 或 HTTPS 服务器。
+description: http-logger 插件将请求和响应日志以 JSON 对象批量推送到 HTTP(S) 服务器，支持自定义日志格式以增强数据管理能力。
---
-
<!--
#
# Licensed to the Apache Software Foundation (ASF) under one or more
@@ -29,89 +28,61 @@ description: 本文介绍了 API 网关 Apache APISIX 的 http-logger 插件。
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/http-logger" />
+</head>
+
## 描述
-`http-logger` 插件可以将 APISIX 的日志数据推送到 HTTP 或 HTTPS 服务器。该插件提供了将日志数据请求作为 JSON 对象发送到监控工具或者其他 HTTP 服务器的功能。
+`http-logger` 插件将请求和响应日志以 JSON 对象批量推送到 HTTP(S) 服务器，并支持自定义日志格式。
## 属性
-| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
-|-------------------------| ------- |-----| ------------- | -------------------- | ------------------------------------------------ |
-| uri | string | 是 | | | HTTP 或 HTTPS 服务器的 URI。 |
-| auth_header | string | 否 | | | 授权 header（如果需要）。 |
-| timeout | integer | 否 | 3 | [1,...] | 发送请求后保持连接处于活动状态的时间。 |
-| log_format | object | 否 | | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
-| include_req_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时，将请求体包含在日志中。如果请求体太大而无法保存在内存中，由于 NGINX 的限制，无法记录。 |
-| include_req_body_expr | array | 否 | | | 当 `include_req_body` 属性设置为 `true` 时的过滤器。只有当此处设置的表达式求值为 `true` 时，才会记录请求体。有关更多信息，请参阅 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 。 |
-| include_resp_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时，包含响应体。 |
-| include_resp_body_expr | array | 否 | | | 当 `include_resp_body` 属性设置为 `true` 时，使用该属性并基于 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 进行过滤。如果存在，则仅在表达式计算结果为 `true` 时记录响应。 |
-| concat_method | string | 否 | "json" | ["json", "new_line"] | 枚举类型： **json**：对所有待发日志使用 `json.encode` 编码。**new_line**：对每一条待发日志单独使用 `json.encode` 编码并使用 `\n` 连接起来。 |
-| ssl_verify | boolean | 否 | false | [false, true] | 当设置为 `true` 时验证证书。 |
+| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
+|------------------------|---------|--------|---------|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| uri | string | True | | | HTTP(S) 服务器的 URI。 |
+| auth_header | string | False | | | HTTP(S) 服务器所需的授权请求头。 |
+| timeout | integer | False | 3 | 大于 0 | 发送请求后保持连接的存活时间。 |
+| log_format | object | False | | | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。也可以通过[插件元数据](../terminology/plugin-metadata.md)在全局范围内配置日志格式，该配置将应用于所有 `http-logger` 插件实例。如果插件实例上的日志格式与插件元数据上的日志格式不同，插件实例的日志格式优先生效。 |
+| include_req_body | boolean | False | false | | 若为 true，则在日志中包含请求体。注意：若请求体太大而无法保存在内存中，由于 NGINX 的限制，将无法记录。 |
+| include_req_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_req_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录请求体。 |
+| include_resp_body | boolean | False | false | | 若为 true，则在日志中包含响应体。 |
+| include_resp_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_resp_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录响应体。 |
+| max_req_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大请求体字节数。超出该值的请求体将被截断。 |
+| max_resp_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大响应体字节数。超出该值的响应体将被截断。 |
+| concat_method | string | False | `json` | `json` 或 `new_line` | 日志的拼接方式。设为 `json` 时对所有待发日志使用 `json.encode`；设为 `new_line` 时也使用 `json.encode`，但用换行符 `\n` 拼接各行。 |
+| ssl_verify | boolean | False | false | | 若为 true，则验证服务器的 SSL 证书。 |
-该插件支持使用批处理器来聚合并批量处理条目（日志和数据）。这样可以避免该插件频繁地提交数据。默认情况下每 `5` 秒钟或队列中的数据达到 `1000` 条时，批处理器会自动提交数据，如需了解更多信息或自定义配置，请参考 [Batch Processor](../batch-processor.md#配置)。
+:::note
-### 默认日志格式示例
+该插件支持使用批处理器来聚合并批量处理条目（日志/数据），避免频繁提交数据。默认情况下，批处理器每 `5` 秒或队列数据达到 `1000` 条时提交数据。详情请参考[批处理器](../batch-processor.md#configuration)。
- ```json
- {
- "service_id": "",
- "apisix_latency": 100.99999809265,
- "start_time": 1703907485819,
- "latency": 101.99999809265,
- "upstream_latency": 1,
- "client_ip": "127.0.0.1",
- "route_id": "1",
- "server": {
- "version": "3.7.0",
- "hostname": "localhost"
- },
- "request": {
- "headers": {
- "host": "127.0.0.1:1984",
- "content-type": "application/x-www-form-urlencoded",
- "user-agent": "lua-resty-http/0.16.1 (Lua) ngx_lua/10025",
- "content-length": "12"
- },
- "method": "POST",
- "size": 194,
- "url": "http://127.0.0.1:1984/hello?log_body=no",
- "uri": "/hello?log_body=no",
- "querystring": {
- "log_body": "no"
- }
- },
- "response": {
- "headers": {
- "content-type": "text/plain",
- "connection": "close",
- "content-length": "12",
- "server": "APISIX/3.7.0"
- },
- "status": 200,
- "size": 123
- },
- "upstream": "127.0.0.1:1982"
- }
- ```
+:::
## 插件元数据
-| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
-| ---------------- | ------- | ------ | ------------- | ------- | ------------------------------------------------ |
-| log_format | object | 否 | | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../../../en/latest/apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
-| max_pending_entries | integer | 否 | | | 在批处理器中开始删除待处理条目之前可以购买的最大待处理条目数。|
+也可以通过配置插件元数据来设置日志格式，可用配置如下：
+
+| 名称 | 类型 | 必选项 | 描述 |
+|---------------------|---------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。 |
+| max_pending_entries | integer | False | 批处理器中允许的最大未处理条目数。达到此限制后，新条目将被丢弃，直到积压减少。在 APISIX 3.15.0 版本中可用。 |
-:::info 注意
+:::info IMPORTANT
-该设置全局生效。如果指定了 `log_format`，则所有绑定 `http-logger` 的路由或服务都将使用该日志格式。
+插件元数据的配置为全局范围生效，将作用于所有使用 `http-logger` 插件的路由和服务。
:::
-以下示例展示了如何通过 Admin API 配置插件元数据：
+## 使用示例
+
+以下示例演示如何在不同场景下配置 `http-logger` 插件。
+
+请先使用 [mockbin](https://mockbin.io) 启动一个模拟 HTTP 日志端点，并记录 mockbin URL。
:::note
-您可以这样从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
+您可以通过以下命令从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
```bash
admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
@@ -119,76 +90,220 @@ admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"/
:::
+### 以默认日志格式记录请求
+
+以下示例演示如何在路由上配置 `http-logger` 插件，记录访问该路由的请求信息。
+
+创建一条路由并配置 `http-logger` 插件，指定服务器 URI：
+
```shell
-curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/http-logger \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr",
- "request": { "method": "$request_method", "uri": "$request_uri" },
- "response": { "status": "$status" }
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/"
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
```
-配置完成后，你将在日志系统中看到如下类似日志：
+向路由发送请求：
```shell
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
+curl "http://127.0.0.1:9080/anything"
+```
+
+您应收到 `HTTP/1.1 200 OK` 响应。在 mockbin 中，您应看到类似如下的日志条目：
+
+```json
+[
+ {
+ "upstream": "3.213.1.197:80",
+ "server": {
+ "hostname": "7d8d831179d4",
+ "version": "3.9.0"
+ },
+ "start_time": 1718291190508,
+ "client_ip": "192.168.65.1",
+ "response": {
+ "status": 200,
+ "headers": {
+ "server": "APISIX/3.9.0",
+ "content-length": "390",
+ "access-control-allow-credentials": "true",
+ "connection": "close",
+ "date": "Thu, 13 Jun 2024 15:06:31 GMT",
+ "access-control-allow-origin": "*",
+ "content-type": "application/json"
+ },
+ "size": 617
+ },
+ "latency": 1200.0000476837,
+ "upstream_latency": 1133,
+ "apisix_latency": 67.000047683716,
+ "request": {
+ "url": "http://127.0.0.1:9080/anything",
+ "querystring": {},
+ "method": "GET",
+ "uri": "/anything",
+ "headers": {
+ "accept": "*/*",
+ "user-agent": "curl/8.6.0",
+ "host": "127.0.0.1:9080"
+ },
+ "size": 85
+ },
+ "service_id": "",
+ "route_id": "http-logger-route"
+ }
+]
```
-## 启用插件
+### 通过插件元数据记录请求和响应头
+
+以下示例演示如何使用[插件元数据](../terminology/plugin-metadata.md)和 NGINX 变量自定义日志格式，记录请求和响应中的特定头部信息。
+
+在 APISIX 中，[插件元数据](../terminology/plugin-metadata.md)用于配置同一插件所有实例的公共元数据字段。当插件在多个资源上启用并需要统一更新元数据字段时，插件元数据非常有用。
-你可以通过如下命令在指定路由上启用 `http-logger` 插件：
+首先，创建一条带有 `http-logger` 插件的路由：
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "plugins": {
- "http-logger": {
- "uri": "http://mockbin.org/bin/:ID"
- }
- },
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/"
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
},
- "uri": "/hello"
-}'
+ "type": "roundrobin"
+ }
+ }'
```
-[mockbin](http://mockbin.org/bin/create) 服务器用于模拟 HTTP 服务器，以方便查看 APISIX 生成的日志。
+接着，为 `http-logger` 配置插件元数据，记录自定义请求头 `env` 和响应头 `Content-Type`：
-## 测试插件
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/http-logger" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "env": "$http_env",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
+```
-你可以通过以下命令向 APISIX 发出请求，访问日志将记录在你的 `mockbin` 服务器中：
+向路由发送带有 `env` 头的请求：
```shell
-curl -i http://127.0.0.1:9080/hello
+curl "http://127.0.0.1:9080/anything" -H "env: dev"
+```
+
+您应收到 `HTTP/1.1 200 OK` 响应。在 mockbin 中，您应看到类似如下的日志条目：
+
+```json
+[
+ {
+ "route_id": "http-logger-route",
+ "client_ip": "192.168.65.1",
+ "@timestamp": "2024-06-13T15:19:34+00:00",
+ "host": "127.0.0.1",
+ "env": "dev",
+ "resp_content_type": "application/json"
+ }
+]
```
-## 删除插件
+### 按条件记录请求体
-当你需要删除该插件时，可以通过如下命令删除相应的 JSON 配置，APISIX 将会自动重新加载相关配置，无需重启服务：
+以下示例演示如何按条件记录请求体。
+
+创建如下带有 `http-logger` 插件的路由，仅当 URL 查询参数 `log_body` 为 `yes` 时才记录请求体：
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "methods": ["GET"],
- "uri": "/hello",
- "plugins": {},
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "http-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "http-logger": {
+ "uri": "https://669f05eb10ca49f18763e023312c3d77.api.mockbin.io/",
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
+```
+
+发送满足条件的带 URL 查询参数的请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
```
+
+您应能看到日志中包含请求体：
+
+```json
+[
+ {
+ "request": {
+ "url": "http://127.0.0.1:9080/anything?log_body=yes",
+ "querystring": {
+ "log_body": "yes"
+ },
+ "uri": "/anything?log_body=yes",
+ "body": "{\"env\": \"dev\"}"
+ }
+ }
+]
+```
+
+不带 URL 查询参数发送请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+此时日志中将不包含请求体。
+
+:::note
+
+若在将 `include_req_body` 或 `include_resp_body` 设为 `true` 的同时自定义了 `log_format`，插件将不会在日志中包含请求体或响应体。
+
+解决方法是在日志格式中使用 NGINX 变量 `$request_body`，例如：
+
+```json
+{
+ "http-logger": {
+ "log_format": {"body": "$request_body"}
+ }
+}
+```
+
+:::
diff --git a/docs/zh/latest/plugins/rocketmq-logger.md b/docs/zh/latest/plugins/rocketmq-logger.md
index c421b148c..fe48e50fd 100644
--- a/docs/zh/latest/plugins/rocketmq-logger.md
+++ b/docs/zh/latest/plugins/rocketmq-logger.md
@@ -5,7 +5,7 @@ keywords:
- API 网关
- Plugin
- RocketMQ
-description: API 网关 Apache APISIX 的 rocketmq-logger 插件用于将日志作为 JSON 对象推送到 Apache RocketMQ 集群中。
+description: rocketmq-logger 插件将请求和响应日志以 JSON 对象批量推送到 RocketMQ 集群，支持自定义日志格式以增强数据管理能力。
---
<!--
@@ -27,202 +27,384 @@ description: API 网关 Apache APISIX 的 rocketmq-logger 插件用于将日志
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/rocketmq-logger" />
+</head>
+
## 描述
-`rocketmq-logger` 插件可以将日志以 JSON 的形式推送给外部 RocketMQ 集群。
+`rocketmq-logger` 插件将请求和响应日志以 JSON 对象批量推送到 RocketMQ 集群，并支持自定义日志格式。
## 属性
-| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
-| ---------------------- | ------- | ------ | ---------------- | ------------- ------- | ------------------------------------------------ |
-| nameserver_list | object | 是 | | | RocketMQ 的 nameserver 列表。 |
-| topic | string | 是 | | | 要推送的 topic 名称。 |
-| key | string | 否 | | | 发送消息的 keys。 |
-| tag | string | 否 | | | 发送消息的 tags。 |
-| log_format | object | 否 | | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
-| timeout | integer | 否 | 3 | [1,...] | 发送数据的超时时间。 |
-| use_tls | boolean | 否 | false | | 当设置为 `true` 时，开启 TLS 加密。 |
-| access_key | string | 否 | "" | | ACL 认证的 Access key，空字符串表示不开启 ACL。 |
-| secret_key | string | 否 | "" | | ACL 认证的 Secret key。 |
-| name | string | 否 | "rocketmq logger" | | 标识 logger 的唯一标识符。如果您使用 Prometheus 监视 APISIX 指标，名称将以 `apisix_batch_process_entries` 导出。 |
-| meta_format | enum | 否 | "default" | ["default"，"origin"] | `default`：获取请求信息以默认的 JSON 编码方式。`origin`：获取请求信息以 HTTP 原始请求方式。更多信息，请参考 [meta_format](#meta_format-示例)。|
-| include_req_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时，包含请求体。**注意**：如果请求体无法完全存放在内存中，由于 NGINX 的限制，APISIX 无法将它记录下来。|
-| include_req_body_expr | array | 否 | | | 当 `include_req_body` 属性设置为 `true` 时进行过滤请求体，并且只有当此处设置的表达式计算结果为 `true` 时，才会记录请求体。更多信息，请参考 [lua-resty-expr](https://github.com/api7/lua-resty-expr)。 |
-| include_resp_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时，包含响应体。 |
-| include_resp_body_expr | array | 否 | | | 当 `include_resp_body` 属性设置为 `true` 时进行过滤响应体，并且只有当此处设置的表达式计算结果为 `true` 时，才会记录响应体。更多信息，请参考 [lua-resty-expr](https://github.com/api7/lua-resty-expr)。 |
+| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
+|------------------------|---------|--------|--------------------|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| nameserver_list | array[string] | True | | | RocketMQ nameserver 列表。 |
+| topic | string | True | | | 要推送数据的目标 topic。 |
+| key | string | False | | | 消息的 key。 |
+| tag | string | False | | | 消息的 tag。 |
+| log_format | object | False | | | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。也可以通过[插件元数据](../terminology/plugin-metadata.md)在全局范围内配置日志格式，将应用于所有 `rocketmq-logger` 插件实例。如果插件实例的日志格式与插件元数据的日志格式不同，插件实例的日志格式优先生效。 |
+| timeout | integer | False | 3 | | 向上游发送数据的超时时间。 |
+| use_tls | boolean | False | false | | 若为 true，则启用 TLS 连接以加密传输。 |
+| access_key | string | False | | | ACL 的 Access key。设置为空字符串将禁用 ACL。 |
+| secret_key | string | False | | | ACL 的 Secret key。 |
+| name | string | False | `rocketmq logger` | | 批处理器中插件的唯一标识符。若使用 Prometheus 监控 APISIX 指标，该名称将以 `apisix_batch_process_entries` 导出。 |
+| meta_format | string | False | `default` | `default` 或 `origin` | 收集请求信息的格式。设为 `default` 以 JSON 格式收集，设为 `origin` 以原始 HTTP 请求格式收集。 |
+| include_req_body | boolean | False | false | | 若为 true，则在日志中包含请求体。注意：若请求体太大而无法保存在内存中，由于 NGINX 的限制，将无法记录。 |
+| include_req_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_req_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录请求体。 |
+| include_resp_body | boolean | False | false | | 若为 true，则在日志中包含响应体。 |
+| include_resp_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_resp_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录响应体。 |
+| max_req_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大请求体字节数。超出该值的请求体将被截断。在 APISIX 3.16.0 版本中可用。 |
+| max_resp_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大响应体字节数。超出该值的响应体将被截断。在 APISIX 3.16.0 版本中可用。 |
+| batch_max_size | integer | False | 1000 | 大于 0 | 每批次允许的最大日志条目数。达到该值后，批次将被发送至日志服务。设置为 1 表示立即处理。 |
+| inactive_timeout | integer | False | 5 | 大于 0 | 在将批次发送至日志服务前等待新日志的最长时间（秒）。该值应小于 `buffer_duration`。 |
+| buffer_duration | integer | False | 60 | 大于 0 | 发送批次前允许最早条目存在的最长时间（秒）。 |
+| retry_delay | integer | False | 1 | 大于等于 0 | 批次发送失败后重试的时间间隔（秒）。 |
+| max_retry_count | integer | False | 0 | 大于等于 0 | 丢弃日志条目前允许的最大重试次数。 |
+
+注意：schema 中还定义了 `encrypt_fields = {"secret_key"}`，这意味着该字段将会被加密存储在 etcd 中。具体参考[加密存储字段](../plugin-develop.md#加密存储字段)。
+
+## 插件元数据
+
+也可以通过配置插件元数据来设置日志格式，可用配置如下：
+
+| 名称 | 类型 | 必选项 | 描述 |
+|---------------------|---------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。 |
+| max_pending_entries | integer | False | 批处理器中允许的最大未处理条目数。达到此限制后，新条目将被丢弃，直到积压减少。在 APISIX 3.15.0 版本中可用。 |
+
+:::info IMPORTANT
+
+插件元数据的配置为全局范围生效，将作用于所有使用 `rocketmq-logger` 插件的路由和服务。
-注意：schema 中还定义了 `encrypt_fields = {"secret_key"}`，这意味着该字段将会被加密存储在 etcd 中。具体参考 [加密存储字段](../plugin-develop.md#加密存储字段)。
+:::
-该插件支持使用批处理器来聚合并批量处理条目（日志/数据）。这样可以避免插件频繁地提交数据，默认设置情况下批处理器会每 `5` 秒钟或队列中的数据达到 `1000` 条时提交数据，如需了解批处理器相关参数设置，请参考 [Batch-Processor](../batch-processor.md#配置)。
+## 使用示例
+
+以下示例演示如何在不同场景下配置 `rocketmq-logger` 插件。
+
+请先启动一个示例 RocketMQ 集群：
+
+```yaml title="docker-compose.yml"
+version: "3"
+
+services:
+ rocketmq_namesrv:
+ image: apacherocketmq/rocketmq:4.6.0
+ container_name: rmqnamesrv
+ restart: unless-stopped
+ ports:
+ - "9876:9876"
+ command: sh mqnamesrv
+ networks:
+ rocketmq_net:
+
+ rocketmq_broker:
+ image: apacherocketmq/rocketmq:4.6.0
+ container_name: rmqbroker
+ restart: unless-stopped
+ ports:
+ - "10909:10909"
+ - "10911:10911"
+ - "10912:10912"
+ depends_on:
+ - rocketmq_namesrv
+ command: sh mqbroker -n rmqnamesrv:9876 -c ../conf/broker.conf
+ networks:
+ rocketmq_net:
+
+networks:
+ rocketmq_net:
+```
-:::tip 提示
+启动容器：
-数据首先写入缓冲区。当缓冲区超过 `batch_max_size` 或 `buffer_duration` 设置的值时，则会将数据发送到 RocketMQ 服务器并刷新缓冲区。
+```shell
+docker compose up -d
+```
-如果发送成功，则返回 `true`。如果出现错误，则返回 `nil`，并带有描述错误的字符串 `buffer overflow`。
+稍等片刻，nameserver 和 broker 应相继启动。
-:::
+创建 `TopicTest` topic：
-### meta_format 示例
+```shell
+docker exec -i rmqnamesrv rm /home/rocketmq/rocketmq-4.6.0/conf/tools.yml
+docker exec -i rmqnamesrv /home/rocketmq/rocketmq-4.6.0/bin/mqadmin updateTopic -n rmqnamesrv:9876 -t TopicTest -c DefaultCluster
+```
-- default:
+等待来自已配置 RocketMQ topic 的消息：
-```json
- {
- "upstream": "127.0.0.1:1980",
- "start_time": 1619414294760,
- "client_ip": "127.0.0.1",
- "service_id": "",
- "route_id": "1",
- "request": {
- "querystring": {
- "ab": "cd"
- },
- "size": 90,
- "uri": "/hello?ab=cd",
- "url": "http://localhost:1984/hello?ab=cd",
- "headers": {
- "host": "localhost",
- "content-length": "6",
- "connection": "close"
- },
- "method": "GET"
- },
- "response": {
- "headers": {
- "connection": "close",
- "content-type": "text/plain; charset=utf-8",
- "date": "Mon, 26 Apr 2021 05:18:14 GMT",
- "server": "APISIX/2.5",
- "transfer-encoding": "chunked"
- },
- "size": 190,
- "status": 200
- },
- "server": {
- "hostname": "localhost",
- "version": "2.5"
- },
- "latency": 0
- }
+```shell
+docker run -it --name rocketmq_consumer -e NAMESRV_ADDR=localhost:9876 --net host apacherocketmq/rocketmq:4.6.0 sh tools.sh org.apache.rocketmq.example.quickstart.Consumer
```
-- origin:
+稍等片刻，消费者应启动并监听来自 APISIX 的消息：
-```http
- GET /hello?ab=cd HTTP/1.1
- host: localhost
- content-length: 6
- connection: close
-
- abcdef
+```text
+Consumer Started.
```
-## 插件元数据设置
+打开新的终端会话，继续以下操作。
-| 名称 | 类型 | 必选项 | 默认值 | 描述 |
-|------------|--------|-----|-------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| log_format | object | 否 | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../../../en/latest/apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
-| max_pending_entries | integer | 否 | | | 在批处理器中开始删除待处理条目之前可以购买的最大待处理条目数。|
+:::note
-:::note 注意
+您可以通过以下命令从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
-该设置全局生效。如果指定了 `log_format`，则所有绑定 `rocketmq-logger` 的路由或服务都将使用该日志格式。
+```bash
+admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
+```
:::
-以下示例展示了如何通过 Admin API 配置插件元数据：
+### 以不同元日志格式记录日志
-:::note
+以下示例演示如何在路由上启用 `rocketmq-logger` 插件，记录客户端请求并推送日志到 RocketMQ，同时了解 `default` 和 `origin` 两种元日志格式的区别。
-您可以这样从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
+创建带 `rocketmq-logger` 的路由，将 `meta_format` 设为 `default`，将 `batch_max_size` 设为 1 以立即发送日志条目：
-```bash
-admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
-:::
+向路由发送请求以生成日志条目：
```shell
-curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/rocketmq-logger \
--H "X-API-KEY: $admin_key" -X PUT -d '
+curl -i "http://127.0.0.1:9080/anything"
+```
+
+您应看到类似如下的日志条目：
+
+```json
{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr",
- "request": { "method": "$request_method", "uri": "$request_uri" },
- "response": { "status": "$status" }
+ "client_ip": "127.0.0.1",
+ "upstream": "34.197.122.172:80",
+ "start_time": 1744727400000,
+ "request": {
+ "headers": {
+ "host": "127.0.0.1:9080",
+ "accept": "*/*",
+ "user-agent": "curl/8.6.0"
+ },
+ "querystring": {},
+ "size": 86,
+ "uri": "/anything",
+ "url": "http://127.0.0.1:9080/anything",
+ "method": "GET"
+ },
+ "route_id": "rocketmq-logger-route",
+ "apisix_latency": 8.9998455047607,
+ "upstream_latency": 503,
+ "latency": 511.99984550476,
+ "response": {
+ "size": 617,
+ "headers": {
+ "content-length": "391",
+ "connection": "close",
+ "date": "Tue, 15 Apr 2025 14:30:00 GMT",
+ "server": "APISIX/3.15.0",
+ "content-type": "application/json"
+ },
+ "status": 200
+ },
+ "server": {
+ "hostname": "apisix",
+ "version": "3.15.0"
+ },
+ "service_id": ""
+}
+```
+
+将 `rocketmq-logger` 的元日志格式更新为 `origin`：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/rocketmq-logger-route" -X PATCH \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "plugins": {
+ "rocketmq-logger": {
+ "meta_format": "origin"
+ }
}
-}'
+ }'
```
-在日志收集处，将得到类似下面的日志：
+再次向路由发送请求以生成新的日志条目：
```shell
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
-{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","request":{"method":"GET","uri":"/hello"},"response":{"status":200},"route_id":"1"}
+curl -i "http://127.0.0.1:9080/anything"
```
-## 启用插件
+您应看到以原始 HTTP 请求格式呈现的日志条目：
+
+```text
+GET /anything HTTP/1.1
+host: 127.0.0.1:9080
+user-agent: curl/8.6.0
+accept: */*
+```
+
+### 通过插件元数据记录请求和响应头
+
+以下示例演示如何使用[插件元数据](../terminology/plugin-metadata.md)和 NGINX 变量自定义日志格式，记录请求和响应中的特定头部信息。
-你可以通过如下命令在指定路由上启用 `rocketmq-logger` 插件：
+在 APISIX 中，[插件元数据](../terminology/plugin-metadata.md)用于配置同一插件所有实例的公共元数据字段。当插件在多个资源上启用并需要统一更新元数据字段时非常有用。
+
+注意：若希望通过插件元数据自定义日志格式，`meta_format` 必须设为 `default`。若 `meta_format` 设为 `origin`，日志条目将保持 `origin` 格式。
+
+首先，创建带 `rocketmq-logger` 的路由，将 `meta_format` 设为 `default`，`batch_max_size` 设为 1 以立即发送日志条目：
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
"plugins": {
- "rocketmq-logger": {
- "nameserver_list" : [ "127.0.0.1:9876" ],
- "topic" : "test2",
- "batch_max_size": 1,
- "name": "rocketmq logger"
- }
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1
+ }
},
"upstream": {
- "nodes": {
- "127.0.0.1:1980": 1
- },
- "type": "roundrobin"
- },
- "uri": "/hello"
-}'
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
+```
+
+接着，为 `rocketmq-logger` 配置插件元数据，记录自定义请求头 `env` 和响应头 `Content-Type`：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/rocketmq-logger" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "env": "$http_env",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
```
-该插件还支持一次推送到多个 `nameserver`，示例如下：
+向路由发送带有 `env` 头的请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -H "env: dev"
+```
+
+您应看到类似如下的日志条目：
```json
-[
- "127.0.0.1:9876",
- "127.0.0.2:9876"
-]
+{
+ "host": "127.0.0.1",
+ "client_ip": "127.0.0.1",
+ "resp_content_type": "application/json",
+ "route_id": "rocketmq-logger-route",
+ "env": "dev",
+ "@timestamp": "2025-04-15T14:30:00+00:00"
+}
```
-## 测试插件
+### 按条件记录请求体
-你可以通过以下命令向 APISIX 发出请求：
+以下示例演示如何按条件记录请求体。
+
+创建如下带 `rocketmq-logger` 的路由，仅当 URL 查询参数 `log_body` 为 `yes` 时才记录请求体：
```shell
-curl -i http://127.0.0.1:9080/hello
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "rocketmq-logger-route",
+ "uri": "/anything",
+ "plugins": {
+ "rocketmq-logger": {
+ "nameserver_list": [ "127.0.0.1:9876" ],
+ "topic": "TopicTest",
+ "key": "key1",
+ "timeout": 30,
+ "meta_format": "default",
+ "batch_max_size": 1,
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
-## 删除插件
+发送满足条件的带 URL 查询参数的请求：
-当你需要删除该插件时，可以通过如下命令删除相应的 JSON 配置，APISIX 将会自动重新加载相关配置，无需重启服务：
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
+```
+
+您应看到日志中包含请求体：
+
+```json
+{
+ "request": {
+ "method": "POST",
+ "body": "{\"env\": \"dev\"}",
+ "size": 183
+ }
+}
+```
+
+不带 URL 查询参数发送请求：
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+此时日志中将不包含请求体。
+
+:::note
+
+若在将 `include_req_body` 或 `include_resp_body` 设为 `true` 的同时自定义了 `log_format`，插件将不会在日志中包含请求体或响应体。
+
+解决方法是在日志格式中使用 NGINX 变量 `$request_body`，例如：
+
+```json
{
- "methods": ["GET"],
- "uri": "/hello",
- "plugins": {},
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
- }
-}'
+ "rocketmq-logger": {
+ "log_format": {"body": "$request_body"}
+ }
+}
```
+
+:::
diff --git a/docs/zh/latest/plugins/syslog.md b/docs/zh/latest/plugins/syslog.md
index ac2acf334..d550517d9 100644
--- a/docs/zh/latest/plugins/syslog.md
+++ b/docs/zh/latest/plugins/syslog.md
@@ -5,7 +5,7 @@ keywords:
- API 网关
- Plugin
- syslog
-description: API 网关 Apache APISIX syslog 插件可用于将日志推送到 Syslog 服务器。
+description: syslog 插件将请求和响应日志以 JSON 对象批量推送到 syslog 服务器，支持自定义日志格式以增强数据管理能力。
---
<!--
@@ -27,60 +27,73 @@ description: API 网关 Apache APISIX syslog 插件可用于将日志推送到 S
#
-->
-## 描述
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/syslog" />
+</head>
-`syslog` 插件可用于将日志推送到 Syslog 服务器。
+## 描述
-该插件还实现了将日志数据以 JSON 格式发送到 Syslog 服务的能力。
+`syslog` 插件将请求和响应日志以 JSON 对象批量推送到 syslog 服务器，并支持自定义日志格式。
## 属性
-| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
-| ---------------- | ------- | ------ | ------------ | ------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
-| host | string | 是 | | | IP 地址或主机名。 |
-| port | integer | 是 | | | 目标上游端口。 |
-| name | string | 否 | "sys logger" | | 标识 logger 的唯一标识符。如果您使用 Prometheus 监视 APISIX 指标，名称将以 `apisix_batch_process_entries` 导出。 |
-| timeout | integer | 否 | 3000 | [1, ...] | 上游发送数据超时（以毫秒为单位）。 |
-| tls | boolean | 否 | false | | 当设置为 `true` 时执行 SSL 验证。 |
-| flush_limit | integer | 否 | 4096 | [1, ...] | 如果缓冲的消息的大小加上当前消息的大小达到（> =）此限制（以字节为单位），则缓冲的日志消息将被写入日志服务器，默认为 4096（4KB）。 |
-| drop_limit | integer | 否 | 1048576 | | 如果缓冲的消息的大小加上当前消息的大小大于此限制（以字节为单位），则由于缓冲区大小有限，当前的日志消息将被丢弃，默认为 1048576（1MB）。 |
-| sock_type | string | 否 | "tcp" | ["tcp","udp"] | 用于传输层的 IP 协议类型。 |
-| max_retry_count | integer | 否 | | [1, ...] | 连接到日志服务器失败或将日志消息发送到日志服务器失败后的最大重试次数。 |
-| retry_delay | integer | 否 | | [0, ...] | 重试连接到日志服务器或重试向日志服务器发送日志消息之前的时间延迟（以毫秒为单位）。 |
-| pool_size | integer | 否 | 5 | [5, ...] | `sock：keepalive` 使用的 Keepalive 池大小。 |
-| log_format | object | 否 | | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
-| include_req_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时包括请求体。 |
-| include_req_body_expr | array | 否 | | | 当 `include_req_body` 属性设置为 `true` 时的过滤器。只有当此处设置的表达式求值为 `true` 时，才会记录请求体。有关更多信息，请参阅 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 。 |
-| include_resp_body | boolean | 否 | false | [false, true] | 当设置为 `true` 时，包含响应体。 |
-| include_resp_body_expr | array | 否 | | | 当 `include_resp_body` 属性设置为 `true` 时进行过滤响应体，并且只有当此处设置的表达式计算结果为 `true` 时，才会记录响应体。更多信息，请参考 [lua-resty-expr](https://github.com/api7/lua-resty-expr)。 |
-
-该插件支持使用批处理器来聚合并批量处理条目（日志/数据）。这样可以避免插件频繁地提交数据，默认情况下批处理器每 `5` 秒钟或队列中的数据达到 `1000` 条时提交数据，如需了解批处理器相关参数设置，请参考 [Batch-Processor](../batch-processor.md#配置)。
-
-### 默认日志格式示例
-
-```text
-"<46>1 2024-01-06T02:30:59.145Z 127.0.0.1 apisix 82324 - - {\"response\":{\"status\":200,\"size\":141,\"headers\":{\"content-type\":\"text/plain\",\"server\":\"APISIX/3.7.0\",\"transfer-encoding\":\"chunked\",\"connection\":\"close\"}},\"route_id\":\"1\",\"server\":{\"hostname\":\"baiyundeMacBook-Pro.local\",\"version\":\"3.7.0\"},\"request\":{\"uri\":\"/opentracing\",\"url\":\"http://127.0.0.1:1984/opentracing\",\"querystring\":{},\"method\":\"GET\",\"size\":155,\"headers\":{\"content-t [...]
-```
+| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
+|------------------------|---------|--------|--------------|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| host | string | True | | | syslog 服务器的 IP 地址或主机名。 |
+| port | integer | True | | | syslog 服务器的目标端口。 |
+| timeout | integer | False | 3000 | 大于 0 | 向上游发送数据的超时时间（毫秒）。 |
+| tls | boolean | False | false | | 若为 true，则验证 TLS。 |
+| flush_limit | integer | False | 4096 | 大于 0 | 推送日志到 syslog 服务器前，缓冲区和当前消息的最大大小（字节，B）。 |
+| drop_limit | integer | False | 1048576 | 大于 0 | 丢弃日志前，缓冲区和当前消息允许的最大大小（字节，B）。 |
+| sock_type | string | False | `tcp` | `tcp` 或 `udp` | 使用的传输层协议。 |
+| pool_size | integer | False | 5 | 大于等于 5 | `sock:keepalive` 使用的连接池大小。 |
+| log_format | object | False | | | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。也可以通过[插件元数据](../terminology/plugin-metadata.md)在全局范围内配置日志格式，将应用于所有 `syslog` 插件实例。如果插件实例的日志格式与插件元数据的日志格式不同，插件实例的日志格式优先生效。 |
+| include_req_body | boolean | False | false | | 若为 true，则在日志中包含请求体。注意：若请求体太大而无法保存在内存中，由于 NGINX 的限制，将无法记录。 |
+| include_req_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_req_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录请求体。 |
+| include_resp_body | boolean | False | false | | 若为 true，则在日志中包含响应体。 |
+| include_resp_body_expr | array | False | | | [lua-resty-expr](https://github.com/api7/lua-resty-expr) 表达式数组。当 `include_resp_body` 为 true 时使用，仅当此处表达式求值为 true 时才记录响应体。 |
+| max_req_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大请求体字节数。超出该值的请求体将被截断。在 APISIX 3.16.0 版本中可用。 |
+| max_resp_body_bytes | integer | False | 524288 | 大于等于 1 | 日志中记录的最大响应体字节数。超出该值的响应体将被截断。在 APISIX 3.16.0 版本中可用。 |
+| name | string | False | `sys logger` | | 批处理器中插件的唯一标识符。若使用 Prometheus 监控 APISIX 指标，该名称将以 `apisix_batch_process_entries` 导出。 |
+| batch_max_size | integer | False | 1000 | 大于 0 | 每批次允许的最大日志条目数。达到该值后，批次将被发送至日志服务。设置为 1 表示立即处理。 |
+| inactive_timeout | integer | False | 5 | 大于 0 | 在将批次发送至日志服务前等待新日志的最长时间（秒）。该值应小于 `buffer_duration`。 |
+| buffer_duration | integer | False | 60 | 大于 0 | 发送批次前允许最早条目存在的最长时间（秒）。 |
+| retry_delay | integer | False | 1 | 大于等于 0 | 批次发送失败后重试的时间间隔（秒）。 |
+| max_retry_count | integer | False | 0 | 大于等于 0 | 丢弃日志条目前允许的最大重试次数。 |
+
+:::note
+
+该插件支持使用批处理器来聚合并批量处理条目（日志/数据），避免频繁提交数据。默认情况下，批处理器每 `5` 秒或队列数据达到 `1000` 条时提交数据。详情请参考[批处理器](../batch-processor.md#configuration)。
+
+:::
## 插件元数据
-| 名称 | 类型 | 必选项 | 默认值 | 描述 |
-|------------|--------|-----|-----|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| log_format | object | 否 | | 日志格式以 JSON 的键值对声明。值支持字符串和嵌套对象（最多五层，超出部分将被截断）。字符串中可通过在前面加上 `$` 来引用 [APISIX 变量](../../../en/latest/apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
+也可以通过配置插件元数据来设置日志格式，可用配置如下：
+
+| 名称 | 类型 | 必选项 | 描述 |
+|------------|--------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------|
+| log_format | object | False | 以 JSON 键值对形式声明的自定义日志格式，值可以引用 [NGINX 变量](https://nginx.org/en/docs/http/ngx_http_core_module.html)。 |
-:::info 重要
+:::info IMPORTANT
-该设置全局生效。如果指定了 `log_format`，则所有绑定 `syslog` 的路由或服务都将使用该日志格式。
+插件元数据的配置为全局范围生效，将作用于所有使用 `syslog` 插件的路由和服务。
:::
-## 启用插件
+## 使用示例
-你可以通过以下命令在指定路由中启用该插件：
+以下示例演示如何在不同场景下配置 `syslog` 插件。
+
+请先在 Docker 中启动一个示例 rsyslog 服务器：
+
+```shell
+docker run -d -p 514:514 --name example-rsyslog-server rsyslog/syslog_appliance_alpine
+```
:::note
-您可以这样从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
+您可以通过以下命令从 `config.yaml` 中获取 `admin_key` 并存入环境变量：
```bash
admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
@@ -88,57 +101,228 @@ admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"/
:::
+### 推送日志到 Syslog 服务器
+
+以下示例演示如何在路由上启用 `syslog` 插件，记录客户端请求并推送日志到 syslog 服务器。
+
+创建带 `syslog` 的路由，将 `host` 和 `port` 替换为您 syslog 服务器的地址和端口，并将 `flush_limit` 设为 1 以立即推送日志：
+
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
"plugins": {
- "syslog": {
- "host" : "127.0.0.1",
- "port" : 5044,
- "flush_limit" : 1
- }
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1
+ }
},
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
+```
+
+向路由发送请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything"
+```
+
+您应收到 `HTTP/1.1 200 OK` 响应。
+
+在 syslog 服务器中，您应看到类似如下的日志条目：
+
+```json
+{
+ "response": {
+ "status": 200,
+ "headers": {
+ "access-control-allow-credentials": "true",
+ "connection": "close",
+ "date": "Sat, 02 Mar 2024 00:14:19 GMT",
+ "access-control-allow-origin": "*",
+ "server": "APISIX/3.8.0",
+ "content-type": "application/json",
+ "content-length": "387"
},
- "uri": "/hello"
-}'
+ "size": 614
+ },
+ "service_id": "",
+ "client_ip": "172.19.0.1",
+ "server": {
+ "hostname": "eff61bf7be4d",
+ "version": "3.8.0"
+ },
+ "upstream": "35.171.123.176:80",
+ "apisix_latency": 13.999900817871,
+ "request": {
+ "method": "GET",
+ "url": "http://127.0.0.1:9080/anything",
+ "querystring": {},
+ "size": 86,
+ "uri": "/anything",
+ "headers": {
+ "host": "127.0.0.1:9080",
+ "accept": "*/*",
+ "user-agent": "curl/7.29.0"
+ }
+ },
+ "route_id": "syslog-route",
+ "upstream_latency": 165,
+ "latency": 178.99990081787,
+ "start_time": 1709334859598
+}
```
-## 测试插件
+### 通过插件元数据自定义日志格式
+
+以下示例演示如何使用[插件元数据](../terminology/plugin-metadata.md)自定义日志格式。插件元数据中配置的日志格式将应用于所有 `syslog` 插件实例。
-现在你可以向 APISIX 发起请求：
+创建带 `syslog` 插件的路由：
```shell
-curl -i http://127.0.0.1:9080/hello
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
+ "plugins": {
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
+ }
+ }'
```
+为 `syslog` 配置插件元数据：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/syslog" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "route_id": "$route_id",
+ "client_ip": "$remote_addr",
+ "resp_content_type": "$sent_http_Content_Type"
+ }
+ }'
```
-HTTP/1.1 200 OK
-...
-hello, world
+
+向路由发送请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything"
```
-## 删除插件
+在 syslog 服务器中，您应看到类似如下的日志条目：
-当你需要删除该插件时，可通过以下命令删除相应的 JSON 配置，APISIX 将会自动重新加载相关配置，无需重启服务：
+```json
+{
+ "@timestamp": "2024-03-02T00:00:31+00:00",
+ "resp_content_type": "application/json",
+ "host": "127.0.0.1",
+ "route_id": "syslog-route",
+ "client_ip": "172.19.0.1"
+}
+```
+
+### 按条件记录请求体
+
+以下示例演示如何按条件记录请求体。
+
+创建如下带 `syslog` 插件的路由，仅当 URL 查询参数 `log_body` 为 `yes` 时才记录请求体：
```shell
-curl http://127.0.0.1:9180/apisix/admin/routes/1 \
--H "X-API-KEY: $admin_key" -X PUT -d '
-{
- "methods": ["GET"],
- "uri": "/hello",
- "plugins": {},
+curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "syslog-route",
+ "uri": "/anything",
+ "plugins": {
+ "syslog": {
+ "host" : "172.0.0.1",
+ "port" : 514,
+ "flush_limit" : 1,
+ "include_req_body": true,
+ "include_req_body_expr": [["arg_log_body", "==", "yes"]]
+ }
+ },
"upstream": {
- "type": "roundrobin",
- "nodes": {
- "127.0.0.1:1980": 1
- }
+ "nodes": {
+ "httpbin.org:80": 1
+ },
+ "type": "roundrobin"
}
-}'
+ }'
```
+
+发送满足条件的带 URL 查询参数的请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything?log_body=yes" -X POST -d '{"env": "dev"}'
+```
+
+您应看到日志中包含请求体：
+
+```json
+{
+ "request": {
+ "method": "POST",
+ "url": "http://127.0.0.1:9080/anything?log_body=yes",
+ "querystring": {
+ "log_body": "yes"
+ },
+ "size": 183,
+ "body": "{\"env\": \"dev\"}",
+ "uri": "/anything?log_body=yes",
+ "headers": {
+ "accept": "*/*",
+ "user-agent": "curl/7.29.0",
+ "host": "127.0.0.1:9080",
+ "content-type": "application/x-www-form-urlencoded",
+ "content-length": "14"
+ }
+ }
+}
+```
+
+不带 URL 查询参数发送请求：
+
+```shell
+curl -i "http://127.0.0.1:9080/anything" -X POST -d '{"env": "dev"}'
+```
+
+此时日志中将不包含请求体。
+
+:::note
+
+若在将 `include_req_body` 或 `include_resp_body` 设为 `true` 的同时自定义了 `log_format`，插件将不会在日志中包含请求体或响应体。
+
+解决方法是在日志格式中使用 NGINX 变量 `$request_body`，例如：
+
+```json
+{
+ "syslog": {
+ "log_format": {"body": "$request_body"}
+ }
+}
+```
+
+:::