You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/01/21 08:13:18 UTC
[GitHub] [apisix] tzssangglass opened a new issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
tzssangglass opened a new issue #3384:
URL: https://github.com/apache/apisix/issues/3384
### Issue description
I used the `kafka-logger` plugin and saw in kafka's monitor that all messages are placed in the same partition by kafka.
I didn't set the `key` parameter of the `kafka-logger` plugin.
### Environment
* apisix version (cmd: `apisix version`):2.2
* OS: (cmd: `uname -a`):centos
* OpenResty / Nginx version: (cmd: `nginx -V` or `openresty -V`)1.19.2
Doubtful places
The `kafka-logger` plugin creates a new producer instance to connect to kafka every time it pushes a message, which causes kafka to reallocate the partition each time, leading to the above phenomenon, please correct me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-765208824
assigned me this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764633660
> > Maybe we can reuse producer via lrucache, like this:
>
> >
>
> > https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67
>
> >
>
> > ?
>
>
>
> yep, I think it's a great way.
Welcome pr!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764481176
Maybe we can reuse producer via lrucache, like this: https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764521849
> Maybe we can reuse producer via lrucache, like this:
>
> https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67
>
> ?
yep, I think it's a great way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764633660
> > Maybe we can reuse producer via lrucache, like this:
>
> >
>
> > https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67
>
> >
>
> > ?
>
>
>
> yep, I think it's a great way.
Welcome pr!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764521849
> Maybe we can reuse producer via lrucache, like this:
>
> https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67
>
> ?
yep, I think it's a great way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #3384:
URL: https://github.com/apache/apisix/issues/3384
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass edited a comment on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
tzssangglass edited a comment on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-765208824
assigned me
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #3384: request help: unbalanced message in kafka partition using kafka-logger plugin
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #3384:
URL: https://github.com/apache/apisix/issues/3384#issuecomment-764481176
Maybe we can reuse producer via lrucache, like this: https://github.com/apache/apisix/blob/e5916cf541281e7fe756dd2e393bd6dcf702c220/apisix/plugins/limit-req.lua#L67?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org