You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "Sn0rt (via GitHub)" <gi...@apache.org> on 2023/04/28 07:08:25 UTC
[GitHub] [apisix] Sn0rt opened a new pull request, #9392: ci: tcp logger with real logger server
Sn0rt opened a new pull request, #9392:
URL: https://github.com/apache/apisix/pull/9392
### Description
#9391
to add the vector as the logger server for receive the log which from user client.
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on pull request #9392: ci: tcp logger with real logger server
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1527191630
> For the content, I see no problem. But I have a question: is there any reason why you chose `vector`?
Simple, easy to configure, well documented.
There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail. There is no problem with how to use `vector `.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on a diff in pull request #9392: ci: tcp logger with real logger server
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on code in PR #9392:
URL: https://github.com/apache/apisix/pull/9392#discussion_r1184779526
##########
t/plugin/tcp-logger.t:
##########
@@ -482,3 +482,119 @@ hello world
the mock backend is hit
--- no_error_log
[error]
+
+
+
+=== TEST 13: true tcp log without tls
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body1 = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "tcp-logger": {
+ "host": "127.0.0.1",
+ "port": 3000,
+ "tls": false,
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1982": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/opentracing"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ local code, _, body2 = t("/opentracing", "GET")
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ ngx.print(body2)
+ }
+ }
+--- request
+GET /t
+--- wait: 0.5
+--- response_body
+opentracing
+
+
+
+=== TEST 14: true tcp log with tls
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body1 = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "tcp-logger": {
+ "host": "127.0.0.1",
+ "port": 43000,
+ "tls": true,
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1982": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/opentracing"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ local code, _, body2 = t("/opentracing", "GET")
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ ngx.print(body2)
+ }
+ }
+--- request
+GET /t
+--- wait: 0.5
+--- response_body
+opentracing
+
+
+
+=== TEST 15: check tcp log
Review Comment:
good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 merged pull request #9392: ci: tcp logger with real logger server
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 merged PR #9392:
URL: https://github.com/apache/apisix/pull/9392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] bzp2010 commented on pull request #9392: ci: tcp logger with real logger server
Posted by "bzp2010 (via GitHub)" <gi...@apache.org>.
bzp2010 commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1527200962
> Simple, easy to configure, well documented.
>
> There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail.
OK, @Sn0rt
For the first point, that's very good. And for the second point, have you tried fluent bit?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on pull request #9392: ci: tcp logger with real logger server
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1535712029
> We need to adjust the existing test cases. Currently, the log is forwarded to port 5044, which is a simulated log service.
copy that. I rewrite some of the tcp-logger test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on pull request #9392: ci: tcp logger with real logger server
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1527219183
> > Simple, easy to configure, well documented.
> > There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail.
>
> OK, @Sn0rt
>
> For the first point, that's very good. And for the second point, have you tried fluent bit (it is the next generation of fluentd)?
I took a brief look at the document construction of fluent-bit, which is not as good as that of vector. My requirement is to cooperate with multiple reporting log formations. fluent-bit does not have an example available out of the box, but vector provides it.
In this Pr, I expect vector to become the basic component of a ci log plug-in test, and to cooperate with other log plug-ins later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] bzp2010 commented on pull request #9392: ci: tcp logger with real logger server
Posted by "bzp2010 (via GitHub)" <gi...@apache.org>.
bzp2010 commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1527250333
@Sn0rt Understand and recognize your position.
But I'm not sure if it's the right strategy to keep using `vector` in the future because essentially the open source project is a vendor-neutral project, the plugin is called `tcp-logger` and customers may use any logging pipeline project that supports TCP, I'm not sure....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on a diff in pull request #9392: ci: tcp logger with real logger server
Posted by "soulbird (via GitHub)" <gi...@apache.org>.
soulbird commented on code in PR #9392:
URL: https://github.com/apache/apisix/pull/9392#discussion_r1181174020
##########
ci/pod/vector/vector_logs_ca.key:
##########
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
Review Comment:
We'd better move certs to `t/certs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on pull request #9392: ci: tcp logger with real logger server
Posted by "soulbird (via GitHub)" <gi...@apache.org>.
soulbird commented on PR #9392:
URL: https://github.com/apache/apisix/pull/9392#issuecomment-1535586770
We need to adjust the existing test cases. Currently, the log is forwarded to port 5044, which is a simulated log service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9392: ci: tcp logger with real logger server
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9392:
URL: https://github.com/apache/apisix/pull/9392#discussion_r1184594804
##########
t/plugin/tcp-logger.t:
##########
@@ -482,3 +482,119 @@ hello world
the mock backend is hit
--- no_error_log
[error]
+
+
+
+=== TEST 13: true tcp log without tls
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body1 = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "tcp-logger": {
+ "host": "127.0.0.1",
+ "port": 3000,
+ "tls": false,
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1982": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/opentracing"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ local code, _, body2 = t("/opentracing", "GET")
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ ngx.print(body2)
+ }
+ }
+--- request
+GET /t
+--- wait: 0.5
+--- response_body
+opentracing
+
+
+
+=== TEST 14: true tcp log with tls
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body1 = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "tcp-logger": {
+ "host": "127.0.0.1",
+ "port": 43000,
+ "tls": true,
+ "batch_max_size": 1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1982": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/opentracing"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ local code, _, body2 = t("/opentracing", "GET")
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("fail")
+ return
+ end
+
+ ngx.print(body2)
+ }
+ }
+--- request
+GET /t
+--- wait: 0.5
+--- response_body
+opentracing
+
+
+
+=== TEST 15: check tcp log
Review Comment:
The test case 15 should be put before the test case 14
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on a diff in pull request #9392: ci: tcp logger with real logger server
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on code in PR #9392:
URL: https://github.com/apache/apisix/pull/9392#discussion_r1184470237
##########
ci/pod/vector/vector_logs_ca.key:
##########
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org