You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "Revolyssup (via GitHub)" <gi...@apache.org> on 2023/05/23 04:59:05 UTC
[GitHub] [apisix] Revolyssup opened a new pull request, #9526: feat: Add real tests for splunk logger using vector
Revolyssup opened a new pull request, #9526:
URL: https://github.com/apache/apisix/pull/9526
### Description
This PR replaces mock splunk-hec-logger tests with real end to end test using vector as log server.
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1201621079
##########
ci/pod/docker-compose.plugin.yml:
##########
@@ -187,18 +187,6 @@ services:
networks:
opa_net:
- # Splunk HEC Logging Service
- splunk:
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1208749840
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -383,12 +361,28 @@ the mock backend is hit
location /t {
content_by_lua_block {
local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "vip": "$remote_addr",
+ "message_3":"test batched data"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
Review Comment:
need a blank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on PR #9526:
URL: https://github.com/apache/apisix/pull/9526#issuecomment-1569650563
> Could you merge the master? The ci always failed because the go packages faile to download.
@monkeyDluffy6017 I merged master. Please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 merged pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 merged PR #9526:
URL: https://github.com/apache/apisix/pull/9526
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on PR #9526:
URL: https://github.com/apache/apisix/pull/9526#issuecomment-1558623490
@Sn0rt please help to review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on PR #9526:
URL: https://github.com/apache/apisix/pull/9526#issuecomment-1569380753
Could you merge the master? The ci always failed because the go packages faile to download.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1208750617
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -224,157 +224,135 @@ hello world
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "message_1":"test custom log format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
uri = "/hello",
upstream = {
type = "roundrobin",
nodes = {
- ["127.0.0.1:1980"] = 1
+ ["127.0.0.1:1982"] = 1
}
},
plugins = {
["splunk-hec-logging"] = {
endpoint = {
- uri = "http://127.0.0.1:1980/splunk_hec_logging",
+ uri = "http://127.0.0.1:18088/services/collector",
token = "BD274822-96AA-4DA6-90EC-18940FB2414C"
},
- batch_max_size = 1,
+ batch_max_size = 3,
inactive_timeout = 1
}
}
- }
- local t = require("lib.test_admin").test
- local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, config)
-
+ })
if code >= 300 then
ngx.status = code
ngx.say(body)
return
end
-
- local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
- ngx.HTTP_PUT,
- [[{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr"
- }
- }]]
- )
+ local code, _, body2 = t("/hello", "GET")
if code >= 300 then
ngx.status = code
- ngx.say(body)
+ ngx.say("fail")
return
end
ngx.say(body)
}
}
+--- wait: 0.5
--- response_body
passed
-=== TEST 8: hit
---- extra_init_by_lua
- local core = require("apisix.core")
- local decode = require("toolkit.json").decode
- local up = require("lib.server")
- up.splunk_hec_logging = function()
- ngx.log(ngx.WARN, "the mock backend is hit")
-
- ngx.req.read_body()
- local data = ngx.req.get_body_data()
- ngx.log(ngx.WARN, data)
- data = decode(data)
- assert(data.event.client_ip == "127.0.0.1")
- assert(data.source == "apache-apisix-splunk-hec-logging")
- assert(data.host == core.utils.gethostname())
- ngx.say('{}')
- end
---- request
-GET /hello
---- wait: 2
---- response_body
-hello world
---- error_log
-the mock backend is hit
---- no_error_log
-[error]
+=== TEST 8: check splunk log
+--- exec
+tail -n 1 ci/pod/vector/splunk.log
+--- response_body eval
+qr/.*test custom log format in plugin.*/
=== TEST 9: set route to test custom log format in route
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "vip": "$remote_addr",
+ "message_2":"logger format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
uri = "/hello",
upstream = {
type = "roundrobin",
nodes = {
- ["127.0.0.1:1980"] = 1
+ ["127.0.0.1:1982"] = 1
}
},
plugins = {
["splunk-hec-logging"] = {
endpoint = {
- uri = "http://127.0.0.1:1980/splunk_hec_logging",
+ uri = "http://127.0.0.1:18088/services/collector",
token = "BD274822-96AA-4DA6-90EC-18940FB2414C"
},
- log_format = {
- host = "$host",
- ["@timestamp"] = "$time_iso8601",
- vip = "$remote_addr"
- },
- batch_max_size = 1,
+ batch_max_size = 3,
inactive_timeout = 1
}
}
- }
- local t = require("lib.test_admin").test
- local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, config)
-
+ })
if code >= 300 then
ngx.status = code
ngx.say(body)
return
end
-
+ local code, _, body2 = t("/hello", "GET")
Review Comment:
need a blank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on PR #9526:
URL: https://github.com/apache/apisix/pull/9526#issuecomment-1563041763
please resolve conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1208918150
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -224,157 +224,135 @@ hello world
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "message_1":"test custom log format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
uri = "/hello",
upstream = {
type = "roundrobin",
nodes = {
- ["127.0.0.1:1980"] = 1
+ ["127.0.0.1:1982"] = 1
}
},
plugins = {
["splunk-hec-logging"] = {
endpoint = {
- uri = "http://127.0.0.1:1980/splunk_hec_logging",
+ uri = "http://127.0.0.1:18088/services/collector",
token = "BD274822-96AA-4DA6-90EC-18940FB2414C"
},
- batch_max_size = 1,
+ batch_max_size = 3,
inactive_timeout = 1
}
}
- }
- local t = require("lib.test_admin").test
- local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, config)
-
+ })
if code >= 300 then
ngx.status = code
ngx.say(body)
return
end
-
- local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
- ngx.HTTP_PUT,
- [[{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr"
- }
- }]]
- )
+ local code, _, body2 = t("/hello", "GET")
if code >= 300 then
ngx.status = code
- ngx.say(body)
+ ngx.say("fail")
return
end
ngx.say(body)
}
}
+--- wait: 0.5
--- response_body
passed
-=== TEST 8: hit
---- extra_init_by_lua
- local core = require("apisix.core")
- local decode = require("toolkit.json").decode
- local up = require("lib.server")
- up.splunk_hec_logging = function()
- ngx.log(ngx.WARN, "the mock backend is hit")
-
- ngx.req.read_body()
- local data = ngx.req.get_body_data()
- ngx.log(ngx.WARN, data)
- data = decode(data)
- assert(data.event.client_ip == "127.0.0.1")
- assert(data.source == "apache-apisix-splunk-hec-logging")
- assert(data.host == core.utils.gethostname())
- ngx.say('{}')
- end
---- request
-GET /hello
---- wait: 2
---- response_body
-hello world
---- error_log
-the mock backend is hit
---- no_error_log
-[error]
+=== TEST 8: check splunk log
+--- exec
+tail -n 1 ci/pod/vector/splunk.log
+--- response_body eval
+qr/.*test custom log format in plugin.*/
=== TEST 9: set route to test custom log format in route
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "vip": "$remote_addr",
+ "message_2":"logger format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
Review Comment:
Added newline at all suggested place @monkeyDluffy6017
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1208747705
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -406,19 +400,23 @@ the mock backend is hit
if code >= 300 then
ngx.status = code
end
- ngx.say(body)
+ local code, _, body2 = t("/hello", "GET")
Review Comment:
need a blank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1201615078
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -406,19 +392,23 @@ the mock backend is hit
if code >= 300 then
ngx.status = code
end
- ngx.say(body)
+ local code, _, body2 = t("/hello", "GET")
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+ ngx.say(passed)
}
}
+--- wait: 0.5
--- response_body
passed
-=== TEST 12: hit
---- pipelined_requests eval
-["GET /hello", "GET /hello", "GET /hello"]
---- wait: 2
+=== TEST 12: check splunk log
+--- exec
+tail -n 1 ci/pod/vector/splunk.log
--- response_body eval
-["hello world\n", "hello world\n", "hello world\n"]
---- no_error_log
-[error]
+qr/.*test batched data.*/
Review Comment:
new line
##########
ci/pod/docker-compose.plugin.yml:
##########
@@ -187,18 +187,6 @@ services:
networks:
opa_net:
- # Splunk HEC Logging Service
- splunk:
Review Comment:
why remove this component?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1201618812
##########
ci/pod/docker-compose.plugin.yml:
##########
@@ -187,18 +187,6 @@ services:
networks:
opa_net:
- # Splunk HEC Logging Service
- splunk:
Review Comment:
Because we don't need a separate splunk server now. Vector does the job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9526: feat: Add real tests for splunk logger using vector
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9526:
URL: https://github.com/apache/apisix/pull/9526#discussion_r1208751404
##########
t/plugin/splunk-hec-logging.t:
##########
@@ -224,157 +224,135 @@ hello world
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "client_ip": "$remote_addr",
+ "message_1":"test custom log format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
uri = "/hello",
upstream = {
type = "roundrobin",
nodes = {
- ["127.0.0.1:1980"] = 1
+ ["127.0.0.1:1982"] = 1
}
},
plugins = {
["splunk-hec-logging"] = {
endpoint = {
- uri = "http://127.0.0.1:1980/splunk_hec_logging",
+ uri = "http://127.0.0.1:18088/services/collector",
token = "BD274822-96AA-4DA6-90EC-18940FB2414C"
},
- batch_max_size = 1,
+ batch_max_size = 3,
inactive_timeout = 1
}
}
- }
- local t = require("lib.test_admin").test
- local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, config)
-
+ })
if code >= 300 then
ngx.status = code
ngx.say(body)
return
end
-
- local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
- ngx.HTTP_PUT,
- [[{
- "log_format": {
- "host": "$host",
- "@timestamp": "$time_iso8601",
- "client_ip": "$remote_addr"
- }
- }]]
- )
+ local code, _, body2 = t("/hello", "GET")
if code >= 300 then
ngx.status = code
- ngx.say(body)
+ ngx.say("fail")
return
end
ngx.say(body)
}
}
+--- wait: 0.5
--- response_body
passed
-=== TEST 8: hit
---- extra_init_by_lua
- local core = require("apisix.core")
- local decode = require("toolkit.json").decode
- local up = require("lib.server")
- up.splunk_hec_logging = function()
- ngx.log(ngx.WARN, "the mock backend is hit")
-
- ngx.req.read_body()
- local data = ngx.req.get_body_data()
- ngx.log(ngx.WARN, data)
- data = decode(data)
- assert(data.event.client_ip == "127.0.0.1")
- assert(data.source == "apache-apisix-splunk-hec-logging")
- assert(data.host == core.utils.gethostname())
- ngx.say('{}')
- end
---- request
-GET /hello
---- wait: 2
---- response_body
-hello world
---- error_log
-the mock backend is hit
---- no_error_log
-[error]
+=== TEST 8: check splunk log
+--- exec
+tail -n 1 ci/pod/vector/splunk.log
+--- response_body eval
+qr/.*test custom log format in plugin.*/
=== TEST 9: set route to test custom log format in route
--- config
location /t {
content_by_lua_block {
- local config = {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/plugin_metadata/splunk-hec-logging',
+ ngx.HTTP_PUT,
+ [[{
+ "log_format": {
+ "host": "$host",
+ "@timestamp": "$time_iso8601",
+ "vip": "$remote_addr",
+ "message_2":"logger format in plugin"
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ local code, body = t('/apisix/admin/routes/1', ngx.HTTP_PUT, {
Review Comment:
need a blank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org