You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/08/23 10:19:40 UTC

[GitHub] [apisix] tzssangglass opened a new pull request #4876: feat(kafka-logger): add cluster name support

tzssangglass opened a new pull request #4876:
URL: https://github.com/apache/apisix/pull/4876


   ### What this PR does / why we need it:
   <!--- Why is this change required? What problem does it solve? -->
   about the cluster_name of kafka producer: https://github.com/doujiang24/lua-resty-kafka#new-1
   <!--- If it fixes an open issue, please link to the issue here. -->
   
   ### Pre-submission checklist:
   
   * [x] Did you explain what problem does this PR solve? Or what new features have been added?
   * [x] Have you added corresponding test cases?
   * [x] Have you modified the corresponding document?
   * [x] Is this PR backward compatible? **If it is not backward compatible, please discuss on the [mailing list](https://github.com/apache/apisix/tree/master#community) first**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r693976352



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       > It confuses me, why name is an integer?
   
   follow the origin design: https://github.com/doujiang24/lua-resty-kafka#new-1
   
   ```
   The third optional cluster_name specifies the name of the cluster, default 1 (yeah, it's number). You can Specifies different names when you have two or more kafka clusters. And this only works with async producer_type.
   ```
   
   I think using numbers should satisfy the need.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r694760826



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -209,7 +209,9 @@ function _M.log(conf, ctx)
     broker_config["producer_type"] = conf.producer_type
 
     local prod, err = core.lrucache.plugin_ctx(lrucache, ctx, nil, create_producer,
-                                               broker_list, broker_config)
+                                               broker_list, broker_config, conf.cluster_name)
+    core.log.info("kafka cluster name ", conf.cluster_name, ", broker_list[1] port ",
+                  prod.client.broker_list[1].port, "\n")

Review comment:
       update

##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
tokers commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r694423639



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       Should we highlight that this is a design in the documents?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander merged pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
spacewander merged pull request #4876:
URL: https://github.com/apache/apisix/pull/4876


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
tokers commented on pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#issuecomment-905400667


   @tzssangglass Please fix the conflicts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r694421261



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -209,7 +209,9 @@ function _M.log(conf, ctx)
     broker_config["producer_type"] = conf.producer_type
 
     local prod, err = core.lrucache.plugin_ctx(lrucache, ctx, nil, create_producer,
-                                               broker_list, broker_config)
+                                               broker_list, broker_config, conf.cluster_name)
+    core.log.info("kafka cluster name ", conf.cluster_name, ", broker_list[1] port ",
+                  prod.client.broker_list[1].port, "\n")

Review comment:
       Why add an additional newline?

##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       Could you add it as a comment?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
tokers commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r693863455



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       It confuses me, why name is an integer?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] membphis commented on a change in pull request #4876: feat(kafka-logger): add cluster name support

Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #4876:
URL: https://github.com/apache/apisix/pull/4876#discussion_r693850880



##########
File path: apisix/plugins/kafka-logger.lua
##########
@@ -60,7 +59,8 @@ local schema = {
         buffer_duration = {type = "integer", minimum = 1, default = 60},
         inactive_timeout = {type = "integer", minimum = 1, default = 5},
         batch_max_size = {type = "integer", minimum = 1, default = 1000},
-        include_req_body = {type = "boolean", default = false}
+        include_req_body = {type = "boolean", default = false},
+        cluster_name = {type = "integer", default = 1},

Review comment:
       Do you need to limit the minimum value?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org