You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2020/12/16 09:45:13 UTC
[GitHub] [apisix] Hcreak opened a new issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Hcreak opened a new issue #3060:
URL: https://github.com/apache/apisix/issues/3060
### Issue description
使用kafka logger插件 输出报错failed to process entries: failed to send data to Kafka topic: not found topic
### Environment
* apisix version (cmd: `apisix version`):
* OS:
### Minimal test code / Steps to reproduce the issue
1.
2.
3.
### What's the actual result? (including assertion message & call stack if applicable)
### What's the expected result?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Firstsawyou commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Firstsawyou commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-750300270
> the `kafka_topic` is an optional field.
>
> @Firstsawyou I think you know this issue, right?
no, `kafka_topic` is a required field.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
membphis commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-747475144
the `kafka_topic` is optional field.
@Firstsawyou I think you know this issue, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
membphis edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-747475144
the `kafka_topic` is an optional field.
@Firstsawyou I think you know this issue, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-748929699
Good to hear.
> But I saw someone encountering the same problem in issues of lua-resty-kafka project
Where is the issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Firstsawyou edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Firstsawyou edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-750300270
> the `kafka_topic` is an optional field.
>
> @Firstsawyou I think you know this issue, right?
no, `kafka_topic` is a required field, and the `key` field is optional.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #3060:
URL: https://github.com/apache/apisix/issues/3060
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-747393019
At least we know the correct topic is sent.
The topic-partition relation is maintained inside lua-resty-kafka, the kafka client used by APISIX. Since the problem may happen after sending topic and before returning from https://github.com/apache/apisix/blob/f476d2c3dc68439406a9531fc8b4a2a2d50c1333/apisix/plugins/kafka-logger.lua#L95-L100, I think you need to add log inside lua-resty-kafka, and see what happened inside.
You can start from https://github.com/doujiang24/lua-resty-kafka/blob/dba4dc13a3e00a587a7b206945bc07ab6ec52345/lib/resty/kafka/client.lua#L144
I will be busy in the coming few days, so I may not reply you for several days.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Hcreak edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Hcreak edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-747159569
Kafka is deployed in k8s Cluster. And I provide a packet capture file with apisix and kafka.
[kafka_loggger.zip](https://github.com/apache/apisix/files/5706534/kafka_loggger.zip)
I found that Kafka gave a response but the gateway did not handle it properly
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-746010542
You should provide either a packet capture file or a runnable example.
Just paste the configuration here isn't a "Minimal test code / Steps to reproduce the issue".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Hcreak commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Hcreak commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-748724678
We set the `ZOOKEEPER_CONNECT` parameter and `ADVERTISED_HOST_NAME` parameter of `kafka` and the `broker_list` parameter of the `kafka_logger` plugin to the form of `IP address`, The problem was solved miraculously.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Hcreak edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Hcreak edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-748724678
We set the `ZOOKEEPER_CONNECT` parameter and `ADVERTISED_HOST_NAME` parameter of `kafka` and the `broker_list` parameter of the `kafka_logger` plugin to the form of `IP address`, The problem was solved miraculously.
I don’t know the specific reason, But I saw someone encountering the same problem under the lua-resty-kafka project.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
spacewander edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-746010542
You should provide either a packet capture file or a runnable example.
Just paste the configuration here isn't a "Minimal test code / Steps to reproduce the issue".
We don't know how you configure the kafka.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Hcreak edited a comment on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Hcreak edited a comment on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-748724678
We set the `ZOOKEEPER_CONNECT` parameter and `ADVERTISED_HOST_NAME` parameter of `kafka` and the `broker_list` parameter of the `kafka_logger` plugin to the form of `IP address`, The problem was solved miraculously.
I don’t know the specific reason, But I saw someone encountering the same problem in issues of lua-resty-kafka project.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Hcreak commented on issue #3060: bug: kafka logger plugin - failed to process entries: failed to send data to Kafka topic: not found topic
Posted by GitBox <gi...@apache.org>.
Hcreak commented on issue #3060:
URL: https://github.com/apache/apisix/issues/3060#issuecomment-747159569
Kafka is deployed in k8s Cluster. And I provide a packet capture file with apisix and kafka.
[kafka_loggger.zip](https://github.com/apache/apisix/files/5706534/kafka_loggger.zip)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org