You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2020/09/07 08:43:26 UTC
[GitHub] [apisix] gy09535 opened a new pull request #2177: WIP:(feature) support send log to aliyun log service
gy09535 opened a new pull request #2177:
URL: https://github.com/apache/apisix/pull/2177
### What this PR does / why we need it:
according to:https://github.com/apache/apisix/issues/2169
### Pre-submission checklist:
* [x] Did you explain what problem does this PR solve? Or what new features have been added?
* [ ] Have you added corresponding test cases?
* [ ] Have you modified the corresponding document?
* [x] Is this PR backward compatible?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r517954144
##########
File path: Makefile
##########
@@ -162,13 +162,15 @@ install: default
$(INSTALL) -d $(INST_LUADIR)/apisix/plugins/zipkin
$(INSTALL) apisix/plugins/zipkin/*.lua $(INST_LUADIR)/apisix/plugins/zipkin/
+
Review comment:
fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-723788684
please review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714854919
> > I do another test , the msg is send with no error ,it is success. And we can check it in sls log console.
>
> can we run it in CI?
The test case can run in CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-729451324
Please review CC @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r520256181
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.warn("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf, ctx)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ctx.var.host,"apisix",
+ ngx.var.pid, conf.project, conf.logstore,
Review comment:
fix it.
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.warn("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
Review comment:
fix it.
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
Review comment:
fix it.
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
Review comment:
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487799703
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
Review comment:
I reference it and change some codes for aliyun.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487798835
##########
File path: conf/config-default.yaml
##########
@@ -185,6 +185,7 @@ plugins: # plugin list
- proxy-cache
- proxy-mirror
- request-id
+ - sls-logger
Review comment:
yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-692061891
> @gy09535 Apache APISIX already has the syslog plugin, can these two plugins be combined into one?
For combined into one I think the aliyun has some special config, we should has different plugin to config it and I reference tcp logger plugin to implement it, I think I can refactor tcp logger and sls logger plugin , change the common code into one module, For system log I think some code is duplicate , for example the batch processor has [retry]( https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L179) the "resty.logger.socket" also has [retry](https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L96) etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-698111106
@liuhengloveyou please take a look at this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
membphis commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714510075
> I do another test , the msg is send with no error ,it is success. And we can check it in sls log console.
can we run it in CI?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on pull request #2177: feat: support logger for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-691917779
@membphis @nic-chen please take a look
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on pull request #2177: feat: support logger for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-691919137
@gy09535 Apache APISIX already has the syslog plugin, can these two plugins be combined into one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518486955
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
+
+local _M = { version = 0.1 }
+
+function _M.encode(facility, severity, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
+ local pri = (Facility[facility] * 8 + Severity[severity])
+ ngx.update_time()
+ local t = os_date(rfc5424_timestamp_format, ngx.now())
+
+ if not hostname then
+ hostname = "-"
+ end
+
+ if not appname then
+ appname = "-"
+ end
+
+ return string_format(rfc5424_format, pri, t, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
Review comment:
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487800878
##########
File path: t/plugin/sls-logger.t
##########
@@ -0,0 +1,174 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 2: missing access_key_secret
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "access_key_secret" is required
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 3: wrong type of string
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your_project", logstore = "your_logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret", timeout = "10"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "timeout" validation failed: wrong type: expected integer, got string
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 4: add plugin
Review comment:
The aliyun log service can not response anything. I can just send it to collector node and then verify it on log service UI, for collector not response anything.I think if the message is send successful with no error log, sending message is success.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518486809
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
Review comment:
fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] idbeta commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
idbeta commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722112241
> @gy09535 you need to resolve the conflict first
ping @gy09535
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-692061891
> @gy09535 Apache APISIX already has the syslog plugin, can these two plugins be combined into one?
For combined into one I think the aliyun has some special config, we should has different plugin to config it and I reference tcp logger plugin to implement it, I think I can refactor tcp logger and sls logger plugin , change the common code into one module, For system log I think some code is duplicate , for example the batch processor has [retry]( https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L179) the "resty.logger.socket" also has [retry](https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L96) etc. so I prefer use cosocket .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r510000720
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
Review comment:
After I check all plugin names us the lowercase names.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495557257
##########
File path: conf/config-default.yaml
##########
@@ -185,6 +185,7 @@ plugins: # plugin list
- proxy-cache
- proxy-mirror
- request-id
+ - sls-logger
Review comment:
sls is the name of aliyun log service, https://www.aliyun.com/product/sls/, should we named "aliyun-sls-logger"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-724419840
style fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495536283
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
Review comment:
I remember the code spec saying, line breaks are not omitted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-725933622
Today Ido a test , I find using string format can costing more cpu , after I change it to string link, it costing little. Here is two picture is before change and after change .
### Test Script
```
./wrk -c 1000 -t 10 -d 60s --latency http://192.168.100.96:9080/whoami
```
###Before
###Aftet
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714343228
> > any update?
>
> you can rebase your branch first. conflicted
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-699591319
> @liuhengloveyou please take a look at this PR
working on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
membphis commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-710052211
> any update?
you can rebase your branch first. conflicted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495541329
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
Review comment:
Review feature first, code style later
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-725933622
Today Ido a test , I find using string format can costing more cpu , after I change it to string link, it costing little. Here is two picture is before change and after change .
### Test Script
```
./wrk -c 1000 -t 10 -d 60s --latency http://192.168.100.96:9080/whoami
```
### Before
### After
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
membphis commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-710071721
@gy09535 I think we need to way to confirm this plugin can work fine with aliyun. if we can check this in the test, it should be great.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518045336
##########
File path: Makefile
##########
@@ -186,6 +177,9 @@ install: default
$(INSTALL) README.md $(INST_CONFDIR)/README.md
$(INSTALL) bin/apisix $(INST_BINDIR)/apisix
+ $(INSTALL) -d $(INST_LUADIR)/apisix/plugins/slslog
Review comment:
last commit fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518041986
##########
File path: Makefile
##########
@@ -162,15 +162,6 @@ install: default
$(INSTALL) -d $(INST_LUADIR)/apisix/plugins/zipkin
$(INSTALL) apisix/plugins/zipkin/*.lua $(INST_LUADIR)/apisix/plugins/zipkin/
-
- $(INSTALL) -d $(INST_LUADIR)/apisix/plugins/skywalking
Review comment:
Why delete these codes?
##########
File path: Makefile
##########
@@ -186,6 +177,9 @@ install: default
$(INSTALL) README.md $(INST_CONFDIR)/README.md
$(INSTALL) bin/apisix $(INST_BINDIR)/apisix
+ $(INSTALL) -d $(INST_LUADIR)/apisix/plugins/slslog
Review comment:
bad indentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518047713
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
Review comment:
we should avoid to concat string in Lua land, it'll make the GC busy.
we should use this style:
`core.log.error("failed to init the socket", soc_err)`
please fix the similar points.
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
+
+local _M = { version = 0.1 }
+
+function _M.encode(facility, severity, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
+ local pri = (Facility[facility] * 8 + Severity[severity])
+ ngx.update_time()
+ local t = os_date(rfc5424_timestamp_format, ngx.now())
+
+ if not hostname then
+ hostname = "-"
+ end
+
+ if not appname then
+ appname = "-"
+ end
+
+ return string_format(rfc5424_format, pri, t, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
Review comment:
bad indentation
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ err_msg = "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ err_msg = "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ core.log.info("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.error("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.error("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ngx.var.host
Review comment:
in APISIX, we shuold use `ctx.var.****` to fetch nginx variable.
here is an example: https://github.com/apache/apisix/blob/master/apisix/plugins/limit-count.lua#L166
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518486955
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
+
+local _M = { version = 0.1 }
+
+function _M.encode(facility, severity, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
+ local pri = (Facility[facility] * 8 + Severity[severity])
+ ngx.update_time()
+ local t = os_date(rfc5424_timestamp_format, ngx.now())
+
+ if not hostname then
+ hostname = "-"
+ end
+
+ if not appname then
+ appname = "-"
+ end
+
+ return string_format(rfc5424_format, pri, t, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
Review comment:
fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r510003753
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
Review comment:
After I check, other plugins do with same code format.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518496377
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
+
+local _M = { version = 0.1 }
+
+function _M.encode(facility, severity, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
+ local pri = (Facility[facility] * 8 + Severity[severity])
+ ngx.update_time()
+ local t = os_date(rfc5424_timestamp_format, ngx.now())
+
+ if not hostname then
+ hostname = "-"
+ end
+
+ if not appname then
+ appname = "-"
+ end
+
+ return string_format(rfc5424_format, pri, t, hostname, appname, pid, project,
+ logstore, access_key_id, access_key_secret, msg)
Review comment:
I do not think is my format is right, I reference this doc: https://github.com/apache/apisix/blob/master/CODE_STYLE.md#fromHistory
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis merged pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis merged pull request #2177:
URL: https://github.com/apache/apisix/pull/2177
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495536938
##########
File path: conf/config-default.yaml
##########
@@ -185,6 +185,7 @@ plugins: # plugin list
- proxy-cache
- proxy-mirror
- request-id
+ - sls-logger
Review comment:
Should the name show aliyun?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518059224
##########
File path: Makefile
##########
@@ -162,15 +162,6 @@ install: default
$(INSTALL) -d $(INST_LUADIR)/apisix/plugins/zipkin
$(INSTALL) apisix/plugins/zipkin/*.lua $(INST_LUADIR)/apisix/plugins/zipkin/
-
- $(INSTALL) -d $(INST_LUADIR)/apisix/plugins/skywalking
Review comment:
last commit fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
moonming commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-724068580
@membphis please check test cases and feature first, we can put code style issues at the end.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714854919
> > I do another test , the msg is send with no error ,it is success. And we can check it in sls log console.
>
> can we run it in CI?
The test case can run in CI, ref: https://github.com/apache/apisix/pull/2177/files#diff-1108dce1e7823ca0b746c4ba69e692e4b02c6d49117d70859954b56cbb810c29R99
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-709949841
any update?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487799703
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
Review comment:
I reference it and change some codes ,this codes is not work for aliyun log service, I fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722294843
> @gy09535 you need to resolve the conflict first
yeah, fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487799703
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
Review comment:
I reference it and change some codes for aliyun.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714343039
> > any update?
>
> you can rebase your branch first. conflicted
> > @gy09535 I think we need to way to confirm this plugin can work fine with aliyun. if we can check this in the test, it should be great.
>
> Thanks I will try it.
To do this first we should have a sls log account and we should complete authorization, I have a company account but I can not share it to "apisix". And also it is not an stability way if sometime the account is deleted. I do another test , the msg is send with no error ,it is success. And we can check it in sls log console.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-724076007
> @membphis please check test cases and feature first, we can put code style issues at the end.
I read the doc and feature before, they are fine. sure about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495536479
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
Review comment:
Use all caps for constants. [CODE_STYLE](https://github.com/apache/apisix/blob/master/CODE_STYLE.md)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495557257
##########
File path: conf/config-default.yaml
##########
@@ -185,6 +185,7 @@ plugins: # plugin list
- proxy-cache
- proxy-mirror
- request-id
+ - sls-logger
Review comment:
sls is the name of aliyun log service, https://www.aliyun.com/product/sls/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518487036
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ err_msg = "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ err_msg = "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ core.log.info("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.error("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.error("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ngx.var.host
Review comment:
fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on a change in pull request #2177: feat: support logger for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487745645
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
Review comment:
Is this file from https://github.com/detailyang/lua-resty-rfc5424/blob/master/lib/resty/rfc5424.lua?
##########
File path: conf/config-default.yaml
##########
@@ -185,6 +185,7 @@ plugins: # plugin list
- proxy-cache
- proxy-mirror
- request-id
+ - sls-logger
Review comment:
Is this plugin only for aliyun SLS?
##########
File path: t/plugin/sls-logger.t
##########
@@ -0,0 +1,174 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 2: missing access_key_secret
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "access_key_secret" is required
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 3: wrong type of string
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your_project", logstore = "your_logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret", timeout = "10"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "timeout" validation failed: wrong type: expected integer, got string
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 4: add plugin
Review comment:
Is there a way to verify the response of Alibaba Cloud?
If not, we can mock Alibaba Cloud
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r523685079
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,104 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
Review comment:
code style:
change to `KERN = LOG_KERN`, which is clearer
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,119 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor.|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero.|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1.|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
Review comment:
bad indentation
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,119 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor.|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero.|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1.|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
Review comment:
ditto
##########
File path: t/plugin/sls-logger.t
##########
@@ -0,0 +1,174 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 2: missing access_key_secret
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "access_key_secret" is required
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 3: wrong type of string
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your_project", logstore = "your_logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret", timeout = "10"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "timeout" validation failed: wrong type: expected integer, got string
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 4: add plugin
Review comment:
we can add a comment in the source code, and mark this plugin is `experiment`. we'll remove this after the community confirm this plugin is stable
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518045893
##########
File path: Makefile
##########
@@ -162,15 +162,6 @@ install: default
$(INSTALL) -d $(INST_LUADIR)/apisix/plugins/zipkin
$(INSTALL) apisix/plugins/zipkin/*.lua $(INST_LUADIR)/apisix/plugins/zipkin/
-
- $(INSTALL) -d $(INST_LUADIR)/apisix/plugins/skywalking
Review comment:
I rebase this file to master branch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722118082
> > @gy09535 you need to resolve the conflict first
>
> ping @gy09535
I will fix it before November 12th
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r523862701
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,104 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
Review comment:
fixed.
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,119 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor.|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero.|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1.|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
Review comment:
fixed.
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,119 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry).|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor.|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero.|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1.|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
Review comment:
fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r487800878
##########
File path: t/plugin/sls-logger.t
##########
@@ -0,0 +1,174 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 2: missing access_key_secret
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your-project", logstore = "your-logstore"
+ , access_key_id = "your_access_key"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "access_key_secret" is required
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 3: wrong type of string
+--- config
+ location /t {
+ content_by_lua_block {
+ local plugin = require("apisix.plugins.sls-logger")
+ local ok, err = plugin.check_schema({host = "cn-zhangjiakou-intranet.log.aliyuncs.com", port = 10009, project = "your_project", logstore = "your_logstore"
+ , access_key_id = "your_access_key", access_key_secret = "your_access_secret", timeout = "10"})
+ if not ok then
+ ngx.say(err)
+ end
+
+ ngx.say("done")
+ }
+ }
+--- request
+GET /t
+--- response_body
+property "timeout" validation failed: wrong type: expected integer, got string
+done
+--- no_error_log
+[error]
+
+
+
+=== TEST 4: add plugin
Review comment:
I think we should mock.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r520254881
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
Review comment:
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495537365
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
Review comment:
and, The commit history should be rebase
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 removed a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 removed a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722118082
> > @gy09535 you need to resolve the conflict first
>
> ping @gy09535
I will fix it before November 12th
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r519852943
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.warn("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
Review comment:
need a simple description?
##########
File path: apisix/plugins/slslog/rfc5424.lua
##########
@@ -0,0 +1,107 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local LOG_EMERG = 0 -- system is unusable
+local LOG_ALERT = 1 -- action must be taken immediately
+local LOG_CRIT = 2 -- critical conditions
+local LOG_ERR = 3 -- error conditions
+local LOG_WARNING = 4 -- warning conditions
+local LOG_NOTICE = 5 -- normal but significant condition
+local LOG_INFO = 6 -- informational
+local LOG_DEBUG = 7 -- debug-level messages
+
+local LOG_KERN = 0 -- kernel messages
+local LOG_USER = 1 -- random user-level messages
+local LOG_MAIL = 2 -- mail system
+local LOG_DAEMON = 3 -- system daemons
+local LOG_AUTH = 4 -- security/authorization messages
+local LOG_SYSLOG = 5 -- messages generated internally by syslogd
+local LOG_LPR = 6 -- line printer subsystem
+local LOG_NEWS = 7 -- network news subsystem
+local LOG_UUCP = 8 -- UUCP subsystem
+local LOG_CRON = 9 -- clock daemon
+local LOG_AUTHPRIV = 10 -- security/authorization messages (private)
+local LOG_FTP = 11 -- FTP daemon
+local LOG_LOCAL0 = 16 -- reserved for local use
+local LOG_LOCAL1 = 17 -- reserved for local use
+local LOG_LOCAL2 = 18 -- reserved for local use
+local LOG_LOCAL3 = 19 -- reserved for local use
+local LOG_LOCAL4 = 20 -- reserved for local use
+local LOG_LOCAL5 = 21 -- reserved for local use
+local LOG_LOCAL6 = 22 -- reserved for local use
+local LOG_LOCAL7 = 23 -- reserved for local use
+
+local Facility = {
+ ["KERN"] = LOG_KERN,
+ ["USER"] = LOG_USER,
+ ["MAIL"] = LOG_MAIL,
+ ["DAEMON"] = LOG_DAEMON,
+ ["AUTH"] = LOG_AUTH,
+ ["SYSLOG"] = LOG_SYSLOG,
+ ["LPR"] = LOG_LPR,
+ ["NEWS"] = LOG_NEWS,
+ ["UUCP"] = LOG_UUCP,
+ ["CRON"] = LOG_CRON,
+ ["AUTHPRIV"] = LOG_AUTHPRIV,
+ ["FTP"] = LOG_FTP,
+ ["LOCAL0"] = LOG_LOCAL0,
+ ["LOCAL1"] = LOG_LOCAL1,
+ ["LOCAL2"] = LOG_LOCAL2,
+ ["LOCAL3"] = LOG_LOCAL3,
+ ["LOCAL4"] = LOG_LOCAL4,
+ ["LOCAL5"] = LOG_LOCAL5,
+ ["LOCAL6"] = LOG_LOCAL6,
+ ["LOCAL7"] = LOG_LOCAL7,
+}
+
+local Severity = {
+ ["EMEGR"] = LOG_EMERG,
+ ["ALERT"] = LOG_ALERT,
+ ["CRIT"] = LOG_CRIT,
+ ["ERR"] = LOG_ERR,
+ ["WARNING"] = LOG_WARNING,
+ ["NOTICE"] = LOG_NOTICE,
+ ["INFO"] = LOG_INFO,
+ ["DEBUG"] = LOG_DEBUG,
+}
+local os_date = os.date
+local ngx = ngx
+local string_format = string.format
+local rfc5424_timestamp_format = "!%Y-%m-%dT%H:%M:%S.000Z"
+local rfc5424_format = "<%d>1 %s %s %s %d - [logservice project=\"%s\" logstore=\"%s\"" ..
+ " access-key-id=\"%s\" access-key-secret=\"%s\"] %s\n"
Review comment:
bad indentation
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "127.0.0.1:1980": 1
+ }
+ },
+ "uri": "/hello"
+}'
+
+## Test Plugin
+
+* success:
+
+```shell
+$ curl -i http://127.0.0.1:9080/hello
+HTTP/1.1 200 OK
+...
+hello, world
+```
+
+* check log in ali cloud log service
+
+
Review comment:
`[****]` please set a title, when failed to load the image, it will show the title
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "127.0.0.1:1980": 1
+ }
+ },
+ "uri": "/hello"
+}'
+
+## Test Plugin
+
+* success:
+
+```shell
+$ curl -i http://127.0.0.1:9080/hello
+HTTP/1.1 200 OK
+...
+hello, world
+```
+
+* check log in ali cloud log service
+
+
+
+## Disable Plugin
+
+Remove the corresponding json configuration in the plugin configuration to disable the `sls-logger`.
+APISIX plugins are hot-reloaded, therefore no need to restart APISIX.
+
+```shell
+$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d value='
+{
+ "methods": ["GET"],
Review comment:
remove this useless line
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.warn("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf, ctx)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ctx.var.host,"apisix",
+ ngx.var.pid, conf.project, conf.logstore,
Review comment:
please fix other similar poitns
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "127.0.0.1:1980": 1
+ }
+ },
+ "uri": "/hello"
+}'
+
+## Test Plugin
+
+* success:
+
+```shell
+$ curl -i http://127.0.0.1:9080/hello
+HTTP/1.1 200 OK
+...
+hello, world
+```
+
+* check log in ali cloud log service
+
+
+
+## Disable Plugin
+
+Remove the corresponding json configuration in the plugin configuration to disable the `sls-logger`.
+APISIX plugins are hot-reloaded, therefore no need to restart APISIX.
+
+```shell
+$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d value='
+{
+ "methods": ["GET"],
Review comment:
and please update the Chinese doc too
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.warn("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf, ctx)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ctx.var.host,"apisix",
+ ngx.var.pid, conf.project, conf.logstore,
Review comment:
please use `ctx.var.***` instead of `ngx.var.***`
`ctx.var.***` is better performance
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
Review comment:
the indentation should be `4` spaces here. please update it
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,206 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ return false, "failed to init the socket" .. soc_err
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ return false, "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ return false, "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port)
+ .. "] err: " .. err
+ end
+
+ core.log.debug("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.warn("failed to set socket keepalive: host[" .. route_conf.host
Review comment:
bad performance, avoid concat string in Lua land.
`core.log.warn("xxxx [", conf.host, "] port[", ...)`, please use this way. and please update other similar points
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 closed pull request #2177: WIP:(feature) support send log to aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 closed pull request #2177:
URL: https://github.com/apache/apisix/pull/2177
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722295037
> > @gy09535 you need to resolve the conflict first
>
> ping @gy09535
work on it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-692061891
> @gy09535 Apache APISIX already has the syslog plugin, can these two plugins be combined into one?
For combined into one I think the aliyun has some special config, we should has different plugin to config it and I reference tcp logger plugin to implement it, I think I can refactor tcp logger and sls logger plugin , change the common code into one module, For syslog I think some code is duplicate , for example the batch processor has [retry]( https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L179) the "resty.logger.socket" also has [retry](https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L96) etc. so I prefer use cosocket .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-714854919
> > I do another test , the msg is send with no error ,it is success. And we can check it in sls log console.
>
> can we run it in CI?
The test case can run in CI ref: https://github.com/apache/apisix/pull/2177/files#diff-1108dce1e7823ca0b746c4ba69e692e4b02c6d49117d70859954b56cbb810c29R99
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-711474875
> @gy09535 I think we need to way to confirm this plugin can work fine with aliyun. if we can check this in the test, it should be great.
Thanks I will try it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] juzhiyuan commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-711778784
Hi, it seems that this PR is still under development, I will convert this one to Draft, feel free to make it ready when needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-722294843
> @gy09535 you need to resolve the conflict first
yeah, fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 edited a comment on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
gy09535 edited a comment on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-692061891
> @gy09535 Apache APISIX already has the syslog plugin, can these two plugins be combined into one?
For combined into one I think the aliyun has some special config, we should has different plugin to config it and I reference tcp logger plugin to implement it, I think I can refactor tcp logger and sls logger plugin , change the common code into one module, For sys-logger plugin I think some code is duplicate , for example the batch processor has [retry]( https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L179) the "resty.logger.socket" also has [retry](https://github.com/apache/apisix/blob/master/apisix/plugins/syslog.lua#L96) etc. so I prefer use cosocket .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] liuhengloveyou commented on a change in pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
liuhengloveyou commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r495536204
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
Review comment:
I remember the code spec saying, line breaks are not omitted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r518486809
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
Review comment:
fix it.
##########
File path: apisix/plugins/sls-logger.lua
##########
@@ -0,0 +1,213 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+local core = require("apisix.core")
+local log_util = require("apisix.utils.log-util")
+local batch_processor = require("apisix.utils.batch-processor")
+local plugin_name = "sls-logger"
+local ngx = ngx
+local rf5424 = require("apisix.plugins.slslog.rfc5424")
+local stale_timer_running = false;
+local timer_at = ngx.timer.at
+local tcp = ngx.socket.tcp
+local buffers = {}
+local tostring = tostring
+local ipairs = ipairs
+local schema = {
+ type = "object",
+ properties = {
+ include_req_body = {type = "boolean", default = false},
+ name = {type = "string", default = "sls-logger"},
+ timeout = {type = "integer", minimum = 1, default= 5000},
+ max_retry_count = {type = "integer", minimum = 0, default = 0},
+ retry_delay = {type = "integer", minimum = 0, default = 1},
+ buffer_duration = {type = "integer", minimum = 1, default = 60},
+ inactive_timeout = {type = "integer", minimum = 1, default = 5},
+ batch_max_size = {type = "integer", minimum = 1, default = 1000},
+ host = {type = "string"},
+ port = {type = "integer"},
+ project = {type = "string"},
+ logstore = {type = "string"},
+ access_key_id = {type = "string"},
+ access_key_secret = {type ="string"}
+ },
+ required = {"host", "port", "project", "logstore", "access_key_id", "access_key_secret"}
+}
+
+local _M = {
+ version = 0.1,
+ priority = 406,
+ name = plugin_name,
+ schema = schema,
+}
+
+function _M.check_schema(conf)
+ return core.schema.check(schema, conf)
+end
+
+local function send_tcp_data(route_conf, log_message)
+ local err_msg
+ local res = true
+ local sock, soc_err = tcp()
+ local can_close
+
+ if not sock then
+ err_msg = "failed to init the socket" .. soc_err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ sock:settimeout(route_conf.timeout)
+ local ok, err = sock:connect(route_conf.host, route_conf.port)
+ if not ok then
+ err_msg = "failed to connect to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ ok, err = sock:sslhandshake(true, nil, false)
+ if not ok then
+ err_msg = "failed to to perform TLS handshake to TCP server: host["
+ .. route_conf.host .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ return false, err_msg
+ end
+
+ core.log.info("sls logger send data ", log_message)
+ ok, err = sock:send(log_message)
+ if not ok then
+ res = false
+ can_close = true
+ err_msg = "failed to send data to TCP server: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err
+ core.log.error(err_msg)
+ else
+ ok, err = sock:setkeepalive(120 * 1000, 20)
+ if not ok then
+ can_close = true
+ core.log.error("failed to set socket keepalive: host[" .. route_conf.host
+ .. "] port[" .. tostring(route_conf.port) .. "] err: " .. err)
+ end
+ end
+
+ if can_close then
+ ok, err = sock:close()
+ if not ok then
+ core.log.error("failed to close the TCP connection, host[",
+ route_conf.host, "] port[", route_conf.port, "] ", err)
+ end
+ end
+
+ return res, err_msg
+end
+
+-- remove stale objects from the memory after timer expires
+local function remove_stale_objects(premature)
+ if premature then
+ return
+ end
+
+ for key, batch in ipairs(buffers) do
+ if #batch.entry_buffer.entries == 0 and #batch.batch_to_process == 0 then
+ core.log.warn("removing batch processor stale object, route id:", tostring(key))
+ buffers[key] = nil
+ end
+ end
+
+ stale_timer_running = false
+end
+
+local function combine_syslog(entries)
+ local data
+ for _, entry in ipairs(entries) do
+ if not data then
+ data = entry.data
+ end
+
+ data = data .. entry.data
+ core.log.info(entry.data)
+ end
+
+ return data
+end
+
+local function handle_log(entries)
+ local data = combine_syslog(entries)
+ if not data then
+ return true
+ end
+
+ -- get the config from entries, replace of local value
+ return send_tcp_data(entries[1].route_conf, data)
+end
+
+-- log phase in APISIX
+function _M.log(conf)
+ local entry = log_util.get_full_log(ngx, conf)
+ if not entry.route_id then
+ core.log.error("failed to obtain the route id for sys logger")
+ return
+ end
+
+ local json_str, err = core.json.encode(entry)
+ if not json_str then
+ core.log.error('error occurred while encoding the data: ', err)
+ return
+ end
+
+ local rf5424_data = rf5424.encode("SYSLOG", "INFO", ngx.var.host
Review comment:
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gy09535 commented on a change in pull request #2177: feat: Support plugin for "aliyun" log service
Posted by GitBox <gi...@apache.org>.
gy09535 commented on a change in pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#discussion_r520256461
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "127.0.0.1:1980": 1
+ }
+ },
+ "uri": "/hello"
+}'
+
+## Test Plugin
+
+* success:
+
+```shell
+$ curl -i http://127.0.0.1:9080/hello
+HTTP/1.1 200 OK
+...
+hello, world
+```
+
+* check log in ali cloud log service
+
+
Review comment:
fix it.
##########
File path: doc/plugins/sls-logger.md
##########
@@ -0,0 +1,118 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+- [中文](../zh-cn/plugins/sls-logger.md)
+
+# Summary
+
+- [**Name**](#name)
+- [**Attributes**](#attributes)
+- [**How To Enable**](#how-to-enable)
+- [**Test Plugin**](#test-plugin)
+- [**Disable Plugin**](#disable-plugin)
+
+## Name
+
+`sls-logger` is a plugin which push Log data requests to ali cloud [Log Server](https://help.aliyun.com/document_detail/112903.html?spm=a2c4g.11186623.6.763.21321b47wcwt1u) with [RF5424](https://tools.ietf.org/html/rfc5424).
+
+This plugin provides the ability to push Log data as a batch to ali cloud log service. In case if you did not recieve the log data don't worry give it some time it will automatically send the logs after the timer function expires in our Batch Processor.
+
+For more info on Batch-Processor in Apache APISIX please refer.
+[Batch-Processor](../batch-processor.md)
+
+## Attributes
+
+|Name |Requirement |Description|
+|--------- |-------- |-----------|
+|host |required | IP address or the Hostname of the TCP server, please reference ali cloud log [Serve List](https://help.aliyun.com/document_detail/29008.html?spm=a2c4g.11186623.2.14.49301b4793uX0z#reference-wgx-pwq-zdb), use IP address insted of domain.|
+|port |required |Target upstream port, default 10009.|
+|timeout |optional |Timeout for the upstream to send data.|
+| project |required|Ali cloud log service project name，please creat in sls before us this plugin.|
+| logstore | required |Ali cloud log service logstore name，please creat in sls before us this plugin.|
+| access_key_id | required | Ali cloud AccessKey ID, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| access_key_secret | required |Ali cloud AccessKey Secret, reference [Authorization](https://help.aliyun.com/document_detail/47664.html?spm=a2c4g.11186623.2.15.49301b47lfvxXP#task-xsk-ttc-ry)。|
+| include_req_body | required| Boolean value. |
+|name |optional |A unique identifier to identity the batch processor|
+|batch_max_size |optional |Max size of each batch.|
+|inactive_timeout|optional |maximum age in seconds when the buffer will be flushed if inactive.|
+|buffer_duration|optional |Maximum age in seconds of the oldest entry in a batch before the batch must be processed.|
+|max_retry_count|optional |Maximum number of retries before removing from the processing pipe line; default is zero|
+|retry_delay |optional |Number of seconds the process execution should be delayed if the execution fails; default is 1|
+
+## How To Enable
+
+The following is an example on how to enable the sls-logger for a specific route.
+
+```shell
+curl http://127.0.0.1:9080/apisix/admin/routes/5 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
+{
+ "plugins": {
+ "sls-logger": {
+ "host": "100.100.99.135",
+ "port": 10009,
+ "project": "your_project",
+ "logstore": "your_logstore",
+ "access_key_id": "your_access_key_id",
+ "access_key_secret": "your_access_key_secret",
+ "timeout": 30000
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "127.0.0.1:1980": 1
+ }
+ },
+ "uri": "/hello"
+}'
+
+## Test Plugin
+
+* success:
+
+```shell
+$ curl -i http://127.0.0.1:9080/hello
+HTTP/1.1 200 OK
+...
+hello, world
+```
+
+* check log in ali cloud log service
+
+
+
+## Disable Plugin
+
+Remove the corresponding json configuration in the plugin configuration to disable the `sls-logger`.
+APISIX plugins are hot-reloaded, therefore no need to restart APISIX.
+
+```shell
+$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d value='
+{
+ "methods": ["GET"],
Review comment:
fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on pull request #2177: feature: support plugin for aliyun log service
Posted by GitBox <gi...@apache.org>.
moonming commented on pull request #2177:
URL: https://github.com/apache/apisix/pull/2177#issuecomment-698111106
@liuhengloveyou please take a look at this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org